0:04 welcome to machine learning full course by simply learn machine learning is changing the world and by 2025 it will be one of the most valuable skills to have companies across Industries whether in healthcare Finance or Tech are looking for experts who can build smart models analyze data and automate complex stocks and the salaries are impressive with machine learning professionals earning between 10 to 25 lakh perom in India and over $120,000 in the US so in this course we'll take you from the basics of machine learning to the most advanced AI techniques you'll also start with an easy follow ml road map tutorials score Concepts like KNN confusion Matrix Q learning and as you progress we'll also dive into the exciting world of deep learning reinforcement learning lstms and we'll also explore the power of large language models hugging phase running LM locally and benchmarking AI model so let's get started but before we comments if you interested and mastering the future of technology the professional certificate course in generative Ai and machine 1:05 learning is the perfect opportunity for you offered in collaboration with the enic Academy I kpur this 11 Monon live and interactive program offers Hands-On expertise in Cutting Edge areas like generative AI machine learning tools like charity di 2 and hugging face so hurry up and enroll now and find the course l in the description box below and in the pin comments hi guys I'm anir vanan from Simply learn and today I will be giving you a tutorial on one one of the most talked about topics in the world of technology and that is machine learning I'm sure that if you are watching this video you must have heard about machine learning somewhere or the other and yes it is quite an intriguing topic this video will act as a supplement for the ones who know the basics of machine learning and are aware of where it is being used and how it is being used for the ones who are completely new to this you don't have to worry we will be brushing up the basics as well so to start off with let's have a look at the agenda here first we'll look into a life without machine learning then we'll move into a life 2:06 with machine learning then we'll try to understand what is machine learning actually then we'll look into the types of machine learning we'll talk about the right machine learning Solutions then we'll move into the different algorithms that machine learning has finally we'll do a use case wherein we'll be predicting the price of a house using linear regression let's talk about our life with without machine learning for someone who doesn't know anything about machine learning or artificial intelligence he or she might think that it is only being used in robots or machines and stuff which is actually true as portrayed by most of the Sci-Fi movies nowadays but you wouldn't believe how much more machine learning is giving us how much of it we are using in our daily lives let's say you need information on any random topic say a computer the first place you would check would definitely be Google right it would collect all the information on the word you search for and present it to you according to relevance if there was no Google you would have to do it in 3:07 a very hard way that is by going through tens or hundreds of books and articles even after which you won't find the answer if we go back 4 to 5 years facial recognition would be a thing only showed in movies like Mission Impossible or Terminator or any of the Sci-Fi movies you pick but machine learning has made it possible for Facebook and Instagram to use this feature for your benefit s you don't have to go through the trouble of tagging every single person present in a picture that you have posted Facebook will automatically recognize the people in the photo and tag them for you that saves a lot of time for you doesn't it also Siri Kotana Iris all of them would not be there to help you if there was no machine learning now that we know life would be a lot more difficult without machine learning let's look into the things that are run by Machine learning and we using them on our daily basis let's enter the world of gaming PS4s and Xboxes have introduced virtual reality glasses which 4:08 brings a whole new level of detail into gaming every time your head Moves In Real World it replicates the movement in the virtual world providing an excellent gaming experience then there is gesture control gaming where in machine learning tracks your body movement and makes a corresponding movement in the game finally in the game FIFA your opponent tends to adapt based on the kind of strategy or gameplay you follow that is again done by Machine learning I'm sure you must have shopped a lot from Amazon so let's look into some of the places Amazon makes use of machine learning let's say you buy a formal t-shirt on Amazon now as you buy it it suggests you formal shoes ties Blazers and apparels that go with what you have bought that's the recommendation system powered by Machine learning again the price on every product that you see on Amazon is varying every moment based on demand that's being done by Machine learning algorithms then comes customer segmentation customer segmentation is 5:09 one of the most crucial thing for all the e-commerce platforms machine learning helps them differentiate between customers based on what they buy how frequent are they and their reviews this helps the companies to make sure that their customers are taking care of and the needs are being fulfilled now let's talk about an app that you use on a daily basis to reach office on time yes it's Uber caps once you have traveled with Uber you must have noticed that it suggests you the places you might want to go based on your previous Journeys when you're taking a share how does the app make sure that the cab you get is traveled by the same route you want to travel in there are so many factors like the distance the traffic the ratings all of it is taken care of by Machine learning for the past 5 minutes we have been talking about how machine learning is improving our lives where all it is being used I have probably said machine learning 100 times already I think it's about time that I tell you what machine learning actually 6:11 is let's take an ordinary system that you currently are using it can't do much except from the basic operations that you already know about now let's add artificial intelligence or for a lay man's point of view let's give the same machine the power to think on its own that is what machine learning basically is it's an application of artificial intelligence that provides systems the ability to learn on their own and improve from experience without being programmed externally so if your computer had machine learning it may be able to play the difficult parts of a game for you or probably solve a complicated mathematical equation for you that could be really helpful let's dive in a little deeper and see how machine Learning Works let's say you provide a system with the input data that carries the photos of various kinds of fruits now you want the system to figure fig out what are the different fruits and group them accordingly so what the system does it analyzes the input data then it tries to find patterns patterns like shapes size and 7:11 color based on these patterns the system will try to predict the different types of fruit and segregate them finally it keeps track of all such decisions it took in the process to make sure it's learning the next time you ask the same system to predict and segregate the different types of fruits it won't have to go through the entire process again that's how machine learning works now let's look into the types of machine learning machine learning is primarily of three types first one is supervised machine learning as the name suggests you have to supervise your machine learning while you train it to work on its own it requires labeled training data next up is unsupervised learning wherein there will be training data but it won't be labeled finally there's reinforce learning wherein the system learns on its own let's talk about all these types in detail let's try to understand how supervised Learning Works look at the pictures very very carefully the monitor depicts the model or the system that we are going to 8:14 train this is how the training is done we provide a data set that contains pictures of a kind of a fruit say an apple then we provide another data set which lets the model know that these pictures were that of a fruit called Apple this ends the training phase now what we will do is we provide a new set of data which only contains pictures of apple now here comes the fun part the system can actually tell you what fruit it is and it will remember this and apply this knowledge in future as well that's how supervised Learning Works you are training the model to do a certain kind of an operation on its own this kind of a model is generally used into filtering spam mails from your email accounts as well yes surpris aren't you so let's move on to unsupervised learning now let's say we have a data set which is cluttered in this case we have a collection of pictures of different fruits we feed this data to the model and the model analyzes the data to figure out patterns in it in the end it 9:16 categorizes the photos into three types as you can see in the image based on their similarities so you provide the data to the system and let the system do the rest of the work simple isn't it this kind of a model is used by flip card to figure out the products that are well suited for you honestly speaking this is my favorite type of machine learning out of all the three and this type has been widely shown in most of the Sci-Fi movies lately let's find out how it works imagine a newborn baby you put a burning candle in front of the baby the baby does not know that if it touches the flame its fingers might get burned so it does that anyway and gets hurt the next time you put that candle in front of the baby it will remember what happened the last time and would not repeat what it did that's exactly how reinforcement learning works we provide the machine with a data set wherein we ask it to identify a particular kind of a fruit in this case an Apple so what it does as a response it tells us that it's 10:16 a mango but as we all know it's a completely wrong answer so as a feedback we tell the system that it's wrong it's not a mango it's an apple what it does it learns from the feedback and keeps that in mind when the next time when we ask a same question it gives us the right answer it is able to tell us that it's actually an apple that is a reinforced response so that's how reinforcement learning works it learns from its mistakes and experiences this model is used in games like Prince of Persia or Assassin's Creed or FIFA where in the level of difficulty increases as you get better with the games just to make it more clear for you let's look at a comparison between supervised and unsupervised learning firstly the data involved in case of supervised learning is labeled as we mentioned in the examples previously we provide the system with a photo of an apple and let the system know that this is actually an apple that is called label data so the system learns from the label data and makes 11:17 future predictions now unsupervised learning does not require any kind of label data because its work is to look for patterns in the input data and organize it the next point is that you get a feedback in case of supervised learning that is once you get the output the system tends to remember that and uses it for the next operation that does not happen for unsupervised learning and the last point is that supervised learning is mostly used to predict data whereas unsupervised learning is used to find out hidden patterns or structures in data I think this would have made a lot of things clear for you regarding supervised and unsupervised learning now let's talk about about a question that everyone needs to answer before building a machine learning model what kind of a machine learning solution should we use yes you should be very careful with selecting the right kind of solution for your model because if you don't you might end up losing a lot of time energy and processing cost I won't be naming 12:17 the actual Solutions because you guys aren't familiar with them yet so we will be looking at it based on supervised unsupervised and reinforcement learning so let's look into the factors that might help us select the right kind of machine learning solution first factor is the problem statement describes the kind of model you will be building or as the name suggests it tells you what the problem is for example let's say the problem is to predict the future stock market prices so for anyone who is new to machine learning would have trouble figuring out the right solution but with time and practice you will understand that for a problem statement like this solution based on supervised learning would work the best for obvious reason re then comes the size quality and nature of the data if the data is cluttered you go for unsupervised if the data is very large and categorical we normally go for supervised learning Solutions finally we choose the solution based on their complexity as for the problem statement 13:18 wherein we predict the stock market prices it can also be solved by using reinforcement learning but that would be very very difficult and time consuming unlike supervised learning algorithms are not types of machine learning in the most simplest language they are methods of solving a particular problem so the first kind of method is classification which falls under supervised learning classification is used when the output you are looking for is a yes or a no or in the form a or b or true or false like if a shopkeeper wants to predict if a particular customer will come back to his shop or not he will use a classification algorithm the algorithms that fall under classification are decision tree knife base random Forest logistic regression and KNN the next kind is regression this kind of a method is used when the predicted data is numerical in nature like if the shopkeeper wants to predict 14:19 the price of a product based on its demand it would go for regression the last method is clustering clustering is a kind of unsupervised learning again it is used when the data needs to be organized most of the recommendation system used by flip cart Amazon Etc make use of clustering another major application of it is in search engines the search engines study your old search history to figure out your preferences and provide you the best search results one of the algorithms that fall under clustering is K me now that we know the various algorithms let's look into four key Al algorithms that are used widely we will understand them with very simple examples the four algorithms that we will try to understand are Kar neighbor linear regression decision tree and knife Pace let's start with our first machine learning solution K nearest neighbor K nearest neighbor is again a 15:21 kind of a classification algorithm as you can see on the screen the similar data points form clusters the blue one the red one and the green one there are three different clusters now if we get a new and unknown data point it is classified based on the cluster closest to it or the most similar to it k in KNN is the number of nearest neighboring data points we wish to compare the unknown data with let's make it clear with an example let's say we have three clusters in a cost to durability graph first cluster is of footballs the second one is of tennis balls and the third one is of basketballs from the graph we can say that the cost of footballs is high and the durability is less the cost of tennis balls is very less but the durability is high and the cost of basketballs is as high as the durability 16:24 now let's say we have an unknown data point we have a black spot which can be one kind of the balls but we don't know what kind it is so what we'll do we'll try to classify this using KNN so if we take K is equal to 5 we draw a circle keeping the unknown data point is the center and we make sure that we have five balls inside that Circle in this case we have a football a basketball and three tennis balls now since we have the highest number of tennis balls inside the circle the classified ball would be a tennis ball so that's how Karis neighbor classification is done linear regression is again a type of supervised learning algorithm this algorithm is used to establish linear relationship between variables one of which would be dependent and the other one would be independent like if we want to predict the weight of a person based on his height weight would be the dependent variable and height would be 17:25 independent let's have a look at it through an example let's say we have a graph here showing a relationship between height and weight of a person let's put the y-axis as H and the x-axis as weight so the green dots are the various data points these green dots are the data points and D is the mean squared eror that is the perpendicular distances from the line to the data points are the error values this error tells us how much the predicted values vary from the original value Let's ignore this blue line for a while so let's say if this is our regression line you can see the distance from all the data points from this line is very high so if we take this line as a regression line the error in the 18:25 prediction will be too high so so in this case the model will not be able to give us a good prediction let's say we draw another regression line here like this even in this case you can see that the perpendicular distance of the data points from the line is very high so the error value will still come as high as the last one so this model will also not be able to give us a good prediction so what to do so finally we draw a line which is this blue line so here we can see that the distance of the data points from the line is very less relative to the other two lines we drew so the value of D for this line will be very less so in this case if we take any value on the x axis the corresponding value on the y- axis will be our prediction and given the fact that the D is very low our prediction should be good 19:26 also this is how reg regression works we draw a line a regression line that is in such a way that the value of D is the least eventually giving us good predictions this algorithm that is decision tree is a kind of an algorithm you can very strongly relate to it uses a kind of a branching method to realize the problem and make decisions based on the conditions let's take this graph as an example imagine yourself sitting at home getting bored you feel like going for a swim what you do is you check if it's sunny outside so that's your first condition if the answer to that condition is yes you go for a swim if it's not Sunny the next question you would ask yourself is if it's raining outside so that's condition number two if it's actually raining you cancel the plan and stay indoors if it's not raining then you would probably go outside and have a walk so that's the final node that's how decision tree algorithm works you probably use this 20:27 every day it real a problem and then takes the decisions based on the answers to every conditions knbis algorithm is mostly used in cases where a prediction needs to be done on a very large data set it makes use of conditional probability conditional probability is the probability of an event say a happening given that another event B has already happened this algorithm is most commonly used in filtering spam mails in your email account let's say you receive a mail the model goes through your old spam maale records then it uses space theorem to predict if the present male is a spam mail or not so p c of a is the probability of even C occurring when a has already occurred p a of C is the probability of event a occurring when C has already occurred and P C is the probability of event C occurring and Pa is the probability of event a occurring 21:28 let's try to understand knife base with a better example knife base can be used to determine on which days to play cricket based on the probabilities of a day being rainy windy or sunny the model tells us if a match is possible if we consider all the weather conditions to be event a for us and the probability of a match being possible event C so the model applies the probabilities of event A and C into the base theorem and predicts if a game of cricket is possible on a particular day or not in this case if the probability of C of a is more than 0.5 we can be able to play a game of cricket if it's less than 0.5 we won't be able to do that that's how n best algorithm works now we have reached the most important part of our video the use case wherein we'll be predicting the price of a house using machine learning look at this guy he seems a bit worried doesn't he let's figure out what the problem here is 22:29 so is's planning on selling his house in a few months but he's unable to decide what price he should sell it for so let's help him predict the price of his house using machine learning so our problem statement is to predict the price of a house using machine learning that will be implemented on python the first job at hand is to figure out which algorithm to use here we will be using linear regression for this problem as I mentioned before linear regression is a kind of a supervised learning let's have a look at the reason why we are choosing linear regression over all the other algorithms so the first reason here is the output is quantitative in nature and directly proportional to the variables so I've mentioned this before many many times that if the prediction is quantitative in nature linear regression is the best choice the next reason is low computation cost what does it mean if you use an algorithm that involves a lot of equation solving and stuff you'll 23:31 need a very powerful system so that will cost you more but not in the case of linear regression because it is a very simple algorithm and it can be done in any normal system finally linear regression is easy to understand as I mentioned before let's have a look at the various features in the data set the first feature is the lower status of population in that area then comes the mean price of all the houses in that area then the Bild building age the tax rate the average number of rooms in every houses the distance to Five Boston Employments accessibility to highways and finally the per capita crime rate so these all are the factors that we'll be taking into consideration to predict the price of a house let's move ahead so let's quickly try to understand how linear regression works so this is your model the computer here and you provide the training data all the values for the different features you provide the system so that it can learn from it as you can see it is trying to study the 24:32 training data it learns from it and then you give it the test data you give it some values that it needs to predict now from the training data that it learned it will be able to predict the price of the house that's how the entire model works it's simple right now that we understand what we are supposed to do let's implement the entire model on python we will be using jupyter notebook to run the code it's because it'll be easier for you to understand and I'll be able to explain every line of it to you properly we'll be using psychic learn to load all the data and libraries psychic learn is a library specially dedicated to machine learning in case you are programming on python so let's start off by importing the required libraries for the model so let's import the various libraries from psyit learn firstly comes up numpy numpy is a support for large arrays and matrices that we are going to use in this use case then then comes pandas pandas is a package that makes working with relational data easier then 25:34 from regression model that we are going to use to check the accuracy of the model we import train test split from scalar doc cross validation import train test split so that's it all the important libraries have been imported now let's move on now to run the cell we press shift plus enter so that's it the cell is running properly now from Psychic learn again we load the data set and to Lo that the code is from sklearn do data sets import load Boston this line is Boston equ Al to load so it makes it easier for us to load the entire data set we 26:36 want so again pressing shift and enter to run the cell no errors so now we'll try to have a look at the data set POS so as you can see the entire data set is available in front of us this is the entire data set this are the values of all the features that we have these are the names of the feature that we talked about before and these are the target values with this value we will compare the predictions that we have so now we'll divide the data into two different data frames this is how we create a data frame DFX this is one of the data frame equal to PD so here's where the Panda's liability come to work b. 27:36 data columns equal to Bon Now features feature names so the next data frame would be dfy to frame boston. Target so in this data frame all the target values will be loaded so that's it we run the cell to it's running fine now let's try to describe the data that is our DFX describe now you can see that this table provides us with the count of every features the mean value for every feature and the standard deviation so all these data can be useful but not in this case so let's not 28:36 waste time here so next comes an important part here we will apply regression now so the code is r equal to linear model do linear regression so that's why we loaded this library in the beginning you press shift plus enter the cell is running pretty well errors now we split our data into train and test data to validate the model as we move ahead I will show you how to do it as well the code is X drain comma X test comma y train comma y test equal to train test split DFX 29:37 dfy test size test size is equal to 0.2 so that means 20% of the data is test and the remaining is trained two so Random state is needed when we are to run and validate our model multiple times so in this case we put it in is equal to 4 so now what we do is we fit the data set into the regression function the code is R do fit X train y train the shift and enter and now as we can see that our data set is fit into the function that we called before in line 14 you can see that so next let's check the coefficient so the code is R do 30:37 CF so these are the weights and these are optimal so let me tell you what coefficient actually is so the coefficients give us an idea of how much the dependent variable will increase if the independent variable value increases by one if you take the equation of line let's say the regression line that is y equal to mx + C so C is the constant and M is the coefficient in this case very simple okay now we will predict the prices so a will be the predicted price so a equal to R do predict X test so we will be predicting the prices according to arrays okay okay so a in Brackets we'll specify the array number so in this case we give three as we press enter we get 31:39 array equal to 18.15 so for array three we are getting 18.1 so if we check on the test data what is the value for array number three this is array number zero this is 1 2 3 for array three the test data is 19.3 and we are getting 18.1 so that's the prediction so you can see that the predicted value and the original value don't vary much so the predictions are pretty good so now let's test for array number five so in the test table array 0 1 2 3 4 5 so in the test the value for the fifth array is 24.5 and we are getting 25.4 so the prediction is still pretty good so our model is functioning pretty well now let's try and find the error so the code will be NP do 32:39 mean in bracket a minus y test so a is the predicted value and Y test is the test value this gives us the error now if we Square this value we get the mean squared error two the shift enter so the error value is 25 so that's our mean squared error so we have successfully created a linear regression model now the error value that we got is 25 which is good enough as you have seen in the predictions it can be made better by manipulating the data as well lesser the mean squared error better the predictions so let me go through the entire process once more for your convenience firstly we'll be importing the various libraries from psyit learn then we load the Boston data set from Psychic learn again then we transfer the data into two different data frames so if we describe the Boston data set we get a table like this 33:40 wherein we get the count mean value and standard deviation for every feature then we train the regression model next up we split the data into test and train wherein the test size is 0.2 and the random state is equal to 4 then we fit the data into the regression model after that we calculate the coefficients and the value of the coefficients you can see on the screen then we try to predict the prices so for array number two we get a value 17.5 and for the same array on the test data it is 17.4 so the difference between the predicted data and the original data is very less a model was functioning pretty well for array number three we get 18.1 and on the test data it is 19.3 so finally we calculate the mean square error which came out to be 25.4 which is quite good for our model to summarize the entire video let's look at a few important things we talked about 34:42 firstly we learned how machine learning is affecting our day-to-day lives with Facebook Flipkart Uber Etc and we understood what machine learning actually is then we talked about the three types of machine learning supervised unsupervised and reinforcement we did a comparison between supervised and unsupervised learning wherein we learned that supervised learning involves label data and in case of unsupervised learning there is non-label data in case of supervised there is direct feedback and in case of unsupervised there is no feedback finally supervised learning is used to predict output and unsupervised learning is used to find hidden structure in a data then we try to understand the four widely used machine learning algorithms that are kous neighbor linear regression decision tree knife base then we learn when to use classification algorithms regression algorithms and clustering algorithms finally we did a use case wherein we predicted the price of a 35:44 house with a mean squared error value of 25.4 which turned out to be pretty fine today we are diving into machine learning the technology behind things like Netflix recommendations CD and even the phase unlock of your phone machine learning helps devices get smarter by learning from data and predicting what we might like or need and here's why machine learning is huge for your career right now machine learning jobs are among the fastest growing roles worldwide companies in every industry Tech healthare Finance and more are looking for people with machine learning skills to improve their products automate tasks and make smarter decisions machine learning engineers in the US earn around $112,000 on on average with plenty of room for growth as you gain experience so if you want to jump into this exciting field learning machine learning can open doors to high paying in demand jobs so in this video I'll guide you through the ultimate road map to Master machine learning in 2025 one step at a 36:44 time so let's get started so in the first month start with the foundations of programming so programming is a language you use to communicate with your computer and bring machine learning algorithms to life so this month is all about python the language of choice for most machine learning practitioners so here's what to focus on first Learn Python Basics begin with Python's fundamentals like variables data types loops and functions so spend time writing small programs daily to get comfortable after that explore the key libraries like numai pandas and psyit learn so numai is for numerical operations it makes handling large data sets faster and easier and and pandas is to manipulate and analyze data so pandas allow you to filter sort and reshape data in a breeze and then psychic learn is for implementing algorithms in just a few lines of code so now you might have heard about R another language using machine learning but don't stress about it now python will serve you well 37:45 especially as a beginner because it's simpler and more flexible so aim to spend an hour or two each day coding by the end of this month you'll have a solid base to build on now in the second month get organized with version control and data structures so this month is about learning how to organize and manage your code effectively and sharpening your problem solving skills with data structures and algorithms so first is version controll with Git so think of git as your project history tracker so imagine working on a big project and making changes then realizing something went wrong you want to go back to an earlier version right so that's where git comes in and here's what you should practice number one is committing changes so save different versions of your work as you progress and then branching which means work on separate features without affecting your main code and then comes merging which means combining changes from different versions once they are ready so you have to set up an account on GitHub or gitlab to store your projects online so not 38:46 only will this be super useful but it'll also start building your portfolio now next is data structures and algorithm so think of data structures like tools in a toolkit so each one like array stxs cues Etc serves a specific purpose so here's how to approach them number one arrays and lists now arrays and lists are for storing data in sequence after that you can get familiar with stacks and cues so stacks and cues are for tasks that need order data access and then you have sorting and searching algorithms so these make your programs more efficient and that's super important in machine learning where data can get massive so the goal here is to build up your problem solving skills which are key to machine learning success so take it slow practice daily and you'll see progress now in the third month learn to access data with SQL so in machine learning a lot of work involves accessing and organizing data from databases so SQL a structured query language is your ticket to getting the data you need for training ml models so here's what you 39:47 should focus on select and wear so these commands help you pull specific pieces of data and then you can move on to joins joins usually combine data from different tables so this is so powerful that you'll use it all the time and then comes Group by and aggregate functions they are great for summarizing data to find patterns so spend time working with sample databases you can find online and practice writing queries being comfortable with SQL will save you time when preparing data for your models now after completing the third month you can move on to mathematics which is building your analytical mind so this month we are tackling the math behind machine learning so don't worry you don't need to be a math genius but understanding certain Concepts will make everything feel less mysterious so in this month you have to focus on linear algebra so this is the math behind how models C data so you can study vectors metrices and operations like multiplication next comes calculus so you use calculus to help your models learn so you have to focus on 40:47 derivatives and gradients which help minimize errors in your model and then you can move on to probability and statistics so understanding probability helps you make sense of data so learn about distributions like normal distribution bormal distribution and then variance and standard deviation so once you have learned maths next you'll be moving on to data handling and visualization which is the heart of machine learning as you all know so with Matt under your belt it's time to dig into Data handling and visualization so data preparation is vital because your model is only as good as the data you feed it so number one comes data manipulation so using pandas and NPI you'll clean and organize your data you might be removing missing values like clean up messy data so it doesn't confuse your model and then you'll learn transforming variables like converting data into formats that work for models and then you will move on to encoding categorical data like changing text Data like female or male into numbers now once you're done with data manipulation next comes data visualization so 41:48 visualization is how you get to see your data before trading a model so here you have to learn mat plot lip and cbon so you can create line charts histograms SC Scatter Plots and heat Maps so this lets you explore patterns and spot outliers so understanding these patterns in your data is crucial for building effective models now in the sixth month you will be moving on to the machine learning fundamentals so now it's time to start building your own models so you'll focus on two main types of machine learning this month number one comes the supervised learning so this is when you train a model on label data where the outcome is already known so you'll work with algorithms like linear regression which predicts a continuous outcome then you'll work with decision trees which breaks down decisions into a tree structure and then you have support Vector machines under supervised learning which updates data into classes now after supervised learning comes unsupervised learning so here your model identifies patterns and data without labeled outcomes so two popular techniques in unsupervised learning is 42:48 number one clustering like K means clustering which means group similar data points and then you have dimensionality reduction this reduces data complexity by focusing on key features so you can use psyit learn to try out these algorithms on Sample data sets so this will give you hands-on experience with model training and you will learn to fine-tune them to get better results now moving on to the seventh month you'll be building and training models with Advanced libraries so by now you have experimented with some basic models so let's step it up with Advanced tools like tensorflow and py torch so these libraries offer more flexibility and power so tensorflow and P torch so here you can start with simp simple models and work your way up so these libraries allow for building neural networks which you'll be studying more on the next month now once you have become familiar with tensa flow and P torch you can move on to model training and evaluation so you have to learn to split data into training and testing sets and evaluate models using metrics like accuracy and PR so your goal this month should be to get comfortable with 43:49 these libraries and understand how they handle data and model training behind the scenes so once you are done with this you'll be moving on to the month where you'll be dealing with Advanced machine learning so this month's concept will be number one on emble learning which means combining multiple models to get better predictions so here you'll be learning about bagging for example random forests here multiple decision trees make predictions and then you have boosting like adab boost XG boost so models learn from each other's mistakes over here and after emble learning comes deep learning so here you explore neural networks which mimic the human so you'll learn about noodle Network Basics so you can start with simple fully connected networks and then you can move on to back propagation and gradient decent so these helps your model learn and improve so you can use tensorflow or pip toch to practice building mural networks so you can work on projects to reinforce these Concepts now moving on you have to specialize on topics like NLP and computer vision so machine learning 44:50 applications are so powerful and here you'll get a taste of two major fields which is NLP or natural language processing so here they work with Text data for tasks like sentiment analysis and text classification so you can start with basic preprocessing like tokenization stop word removal and move to building simple NLP models after that you can try computer vision so for image data you have to learn CNN convolutional neural networks so these networks analyze visual patterns making them ideal for image classification so you practice with open data sets like text documents or images and apply the concepts you will learn to see results in real world applications now in the 10th month you'll be dealing with model deployment which is bringing your models to life so here you'll be using flask or Jango so you can use these Frameworks to create a web API so users can interact with your model for example build a web app that lets people upload images for classification and then you can also try Docker so package your model and its dependency so it can run on any machine 45:52 so this is super helpful for deploying models without compatibility issues so by the end of this month you'll be able to share your models with the world so moving on to the 11th month you'll be starting with cloud and production so this month you'll learn how to deploy models on the cloud and ensure they perform well in real world environments so you'll be dealing with Cloud platforms like AWS Google cloud or Azure so you have to learn to deploy models of the cloud for wider accessibility and scalability and then comes monitoring and maintenance so understand how to track your model's performance over time and update it as needed so these skills are essential for maintaining models and production and ensuring this you're reliable and finally you will be creating real world projects and portfolio building so here you have to choose topics that interest you and showcase your skills so first you can start with full projects so complete projects that go from data cleaning and model building to deployment so ideas could be a sentiment analysis tool or an image recognition app and then you have to build your portfolio so organize and 46:53 document your projects host them on GitHub and create an online port portfolio to share with potential employers or collaborators so by following this road map you'll be well prepared to handle real world machine learning challenges and have an impressive portfolio to show for it so thank you so much for joining me on this ultimate do map to mastering machine learning in 2025 I hope this month by Monon guide gives you a clear achievable path to dive into machine learning from the basics to deploying your own models remember learning machine learning is a journey there will be challenges along the way but with patience and consist practice you'll see progress step by step what's in it for you we're going to cover clustering what is clustering K means clustering which is one of the most common used clustering tools out there including a flowchart to understand K means clustering and how it functions and then we'll do an actual python live demo on clustering of cars based on Brands then we're going to cover logistic regression what is logistic regression logistic regression 47:54 curve in sigmoid function and then we'll do a another python code demo to classify a tumor as malignant or benign based on features and let's start with clustering suppose we have a pile of books of different genres now we divide them into different groups like fiction horror education and as we can see from this young lady she definitely is into heavy you can just tell by those eyes in the maple Canadian leaf on her shirt but we have fiction horror and education and we want to go ahead and divide our books up well organizing objects into grp groups based on similarity is clustering and in this case as we're looking at the books we're talking about clustering things with no one categories but you can also use it to explore data so you might not know the categories you just know that you need to divide it up in some way to conquer the data and to organize it better but in this case uh we're going to be looking at clustering in specific categories and let's just take a deeper look at that we're going to use K means clustering K means clustering is probably the most commonly used 48:55 clustering tool in the machine learning library K means clustering is an example of unsupervised learning if you remember from our previous thing it is used when you have unlabeled data so we don't know the answer yet we have a bunch of data that we want to Cluster into different groups Define clusters in the data based on feature similarity so we've introduced a couple terms here we've already talked about unsupervised learning and unlabeled data so we don't know the answer yet we're just going to group stuff together and see if we can find an un answer of how things connect we've also introduced feature similarity features being different features of the data now with books we can easily see fiction and horror and history books but a lot of times with data some of that information isn't so easy to see right when we first look at it and so K means is one of those tools where we can start finding things that connect that match with each other suppose we have these data points and want to assign them into a cluster now when I look at these data 49:55 points I would probably group them into two clusters just by looking at them I'd say two of these group of data kind of come together but in K means we pick K clusters and assign random centroids to clusters where the K clusters represents two different clusters we pick K clusters inside random centroids to the Clusters then we compute distance from objects to the centroids now we form new clusters based on minimum distances and calculate the centroids so we figure out what the best distance is for the centroid then we move the centroid and recalculate those distances repeat previous two steps iteratively till the cluster centroid stop changing their positions and become Static repeat previous two steps iteratively till the cluster centroid stop changing and the positions become Static once the Clusters become Static then K means clustering algorithm is said to be converged and there's another term we see throughout machine learning is converged that means whatever math we're using to figure out the answer has come to a solution or it's it's conversed on 50:57 an answer shall we see the flowchart to understand make a little bit more sense by putting it into a nice easy step by step so we start we choose K we'll look at the elbow method in just a moment we assign random centroids to clusters and sometimes you pick the centroids because you might look at the data in in a graph and say oh these are probably the central points then we compute the distance from the objects to the centroids we take that and we form new clusters based on minimum distance and calculate their centroids then we compute the distance from objects to the new centroids and then we go back and repeat those last two steps we calculate the distances so as we're doing it it brings into the new centroid and then we move the centroid around and we figure out what the best which objects are closest to each centroid so the objects can switch from one centroid to the other as the centroids are moved around and we continue that until it is converged let's see an example of this suppose we have this data set of seven individual ual and their score on two 51:58 topics A and B so here's our subject in this case referring to the person taking the uh test and then we have subject a where we see what they've scored on their first subject and we have subject B and we can see what they score on the second subject now let's take two farthest apart points as initial cluster centroids now remember we talked about selecting them randomly or we can also just put them in different points and pick the furthest one apart so they move together either one works okay depending on what kind of data you're working on and what you know about it so we took the two furthest points one and one and five and 7even and now let's take the two farthest apart points as initial cluster centroids each point is then assigned to the closest cluster with respect to the distance from the centroids so we take each one of these points in there we measure that distance and you can see that if we measured each of those distances and you use the Pythagorean theorem for a triangle in this case because you know the X and the Y and you can figure out the uh diagonal 52:58 line from that or you just take a ruler and put it on your monitor that'd be kind of silly but it would work if you're just eyeballing it you can see how they naturally come together in certain areas now we again calculate the centroids of each cluster so cluster one and then cluster two and we look at each individual dot there's one two three we're in one cluster uh the centroid then moves over it becomes 1.8 comma 2.3 so remember it was that one in one well the very center of the data we're looking at would put it at the one point roughly 22 but 1.8 and 2.3 and the second one if we wanted to make the overall mean Vector the average Vector of all the different distances to that centroid we come up with 4 comma 1 and 54 so we've now moveed the centroids We compare each individual's distance to its own cluster mean and to that of the opposite cluster and we find can build a nice chart on here that the as we move that Centro around we now have a new different kind of cluster Str ing of groups and using ukian distance between 53:59 the points and the mean we get the same formula you see new formulas coming up so we have our individual dots distance to the mean centr of the cluster and distance to the mean Central of the cluster only individual three is nearer to the mean of the opposite cluster cluster two than its own cluster one and you can see here in the diagram where we've kind of circled that one in the middle so when we've moved the clust the centroids of the Clusters over one of the points shifted to the other cluster because it's closer to that group of individuals thus individual 3 is relocated to Cluster 2 resulting in a new Partition and we regenerate all those numbers of how close they are to the different clusters for the new clusters we will find the actual cluster centroids so now we move the centroids over and you can see that we've now formed two very distinct clusters on here on comparing the distance of each individual's distance to its own cluster mean and to that of the opposite cluster we find that the data points are stable hence we have our final cluster now if you remember I brought up a concept earlier k mean on the K means 55:01 algorithm choosing the right value of K will help in less number of iterations and to find the appropriate number of clusters in a data set we use the elbow method and within sum of squares WSS is defined as the sum of the squared distance between each member of the cluster and its centroid and so you see we've done here is we have the number of clusters and as you do the same k means algorithm over the different clusters and you calculate what that centroid looks like and you find the optimal you can actually find the optimal number of clusters using the elbow of the graph is called as the elbow method and on this we guessed at two just by looking at the data but as you can see the slope you actually just look for right there where the elbow is in the slope and you have a clear answer that we want two different to start with k means equals 2 A lot of times people end up Computing K means equals 2 3 4 5 until they find the value which fits on the elbow joint sometimes you can just look at the data and if you're really good with that specific 56:02 domain remember domain I mentioned that last time you'll know that that where to pick those numbers or where to start guessing at what that K value is so let's take this and we're going to use a use case using K means clustering to Cluster cars into Brands using parameters such as horsepower cubic inches make year Etc so we're going to use the data set cars data having information about three brands of cars Toyota Honda and Nissan we'll go back to my favorite tool the Anaconda Navigator with the Jupiter notebook and let's go ahead and flip over to our Jupiter notebook and in our Jupiter notebook I'm going to go ahead and just paste the uh basic code that we usually start a lot of these off with we're not going to go too much into this code because we've already discussed numpy we've already discussed met plot library and pandas that be being the number array pandas being the pandas data frame and M plot for the graphing and don't forget uh since if you're using the Jupiter notebook you do need the matap plot library in line so that it plots 57:03 everything on the screen if you're using a different python editor then you probably don't need that because it'll have a popup window on your computer and we'll go ahead and run this just to load our libraries and our setup into here the next step is of course to look at our data which I've already opened up in a spreadsheet and you can see here we have the miles per gallon cylinders inches horsepower weight pounds how you how heavy it is time it takes to get to 60 my card is probably on this one at about 80 or 90 what year it is so this is you can actually see this is kind of older cars and then the brand Toyota Honda Nissan so the different cars are coming from all the way from 1971 if we scroll down to uh the 80s we have between the 70s and 80s a number of cars that they've put out and let's uh when we come back here we're going to do importing the data so we'll go ahead and do data set equals and we'll use pandas to read this in and it's a from a CSV file remember you can always post this 58:03 in the comments and request the data files for these either in the comments here on the YouTube video or go to Simply learn.com and request that the car CSV I put it in the same folder as the code that I've stored so my python code is stored in the same folder so I don't have to put the full path if you store them in different folders you do have to change this and double check your name variables and we'll go ahead and run this and uh We've chosen data set arbitrarily because you know it's a data set we're importing and we've now imported our car CSV into the data set as you know you have to prep the data so we're going to create the X data this is the one that we're going to try to figure out what's going on with and then there is a number of ways to do this but we'll do it in a simple Loop so you can actually see what's going on so we'll do for I and x. columns so we're going to go through each of the columns and a lot of times it's important I I'll make lists of the columns and do this because I might remove certain columns or there might be columns that I want to be 59:04 processed differently but for this we can go ahead and take X of I and we want to go fill na a and that's a panda command but the question is what are we going to fill the missing data with we definitely don't want to just put in a number that doesn't actually mean something and so one of the tricks you can do with this is we can take X of I and in addition to that we want to go ahead and turn this into an integer because a lot of these are integers so we'll go ahead and keep it integers and me add the bracket here and a lot of editors will do this they'll think that you're closing one bracket make sure you get that second bracket in there if it's a double bracket that's always something that happens regularly so once we have our integer of X ofi this is going to fill in any missing data with the average and I was so busy closing one set of brackets I forgot that the mean is also has brackets in there for the pandas so we can see here we're going to fill in all the data with the average value for that column so if there's missing data in the average of the data it does have then once we've done that 1:00:04 we'll go ahead and loop through it again and just check and see to make sure everything is filled in correctly and we'll print and then we take X is null and this returns a set of the null value or the how many lines are null and we'll just sum that up to see what that looks like and so when I run this and so with the X what we want to do is we want to remove the last column because that had the models that's what we're trying to see if we can cluster these things and figure out the models there is so many different ways to sort the X out for one we could take the X and we could go data set our variable we're using and use the iocation one of the features that's in pandas and we could take that and then take all the rows and all but the last column of the data set and at this time we could do values we just convert it to values so that's one way to do this and if I let me just put this down here and print X it's a capital x we chose and I 1:01:05 run this you can see it's just the values we could also take out the values and it's not going to return anything because there's no values connected to it what I like to do with this is instead of doing the iocation which does integers more common is to come in here and we have our data set and we're going to do data set dot or data set. columns and remember that lists all the columns so if I come in here let me just Mark that as red and I print data set. columns you can see that I have my index here I have my MPG cylinders everything including the brand which we don't want so the way to get rid of the brand would be to do data Columns of Everything But the last one minus one so now if I print this you'll see the brand disappears and so I can actually just take data set columns minus one and I'll put it right in here for 1:02:06 the columns we're going to look at and let's unmark this and unmark this and now if I do an x. head I now have a new data frame and you can see right here we have all the different columns except for the brand at the end of the year and it turns out when you start playing with the data set you're going to get an error later on and it'll say cannot convert string to uh float value and that's because for some reason these things the way they recorded them must have been recorded as strings so we have a neat feature in here on pandas to convert and it is simply convert objects and for this we're going to do convert oops convert underscore numeric numeric equals true and yes I did have to go look that up I don't have it memorized the convert numeric in there if I'm working with a lot of these things I remember them but um depending 1:03:08 on where I'm at what I'm doing I usually have to look it up and we run that oops I must have missed something in here let me double check my spelling and when I double check my spelling you'll see I missed the first underscore in the convert objects when I run this it now has everything converted into a numeric value because that's what we're going to be working with is numeric values down here and the next part is that we need to go through the data and eliminate null values most people when they're doing small amounts you're working with small data pools discover afterwards that they have a null value and they have to go back and do this so you know be aware whenever we're formatting this data things are going to pop up and sometimes you go backwards to fix it and that's fine that's just part of exploring the data and understanding what you have and I should have done this earlier but let me go ahead and increase the size of my window one notch there we go easier to see so we'll do 4 I in working with x. 1:04:10 column so we'll page through all the columns and we want to take X of I we're going to change that we're going to alter it and so with this we want to go ahead and fill in X of I pandis Has the fill in a and that just fills in any non-existent missing data and we'll put my brackets up and there's a lot of different ways to fill this data if you have a really large data set some people just void out that data because if and then look at it later in a separate uh exploration of data one of the tricks we can do is we can take our column and we can find the means and the means is in our quotation marks when we take The Columns we going to fill in the the non-existing one with the means the problem is that returns a decimal float so some of these aren't decimals certainly need to be a little careful of doing this but for this example we're just going to fill it in with the integer version of this keeps 1:05:12 it on par with the other data that isn't a decimal point and then what we also want to do is we want to double check A lot of times you do this first part first to double check then you do the fill and then you do it again just to make sure you did it right so we're going to go through and test for missing data and one of the ways you can do that is simply go in here and take our X ofi column so it's going to go through the x of I column it says is null so it's going to return any any place there's a null value it actually goes through all the rows of each column is null and then we want to go ahead and sum that so we take that we add the sum value and these are all pandas so so is n is a panda command and so is Su and if we go through that and we go ahead and run it and we go ahead and take and run that you'll see that all the columns have zero null values so we've now tested and double checked and our data is nice and clean we have no null values everything 1:06:13 is now a number value we turned it into numeric and we've removed the last column in our data and at this point we're actually going to start using the elbow method to find the optimal number number of clusters so we're now actually getting into the SK learn part uh the K means clustering on here I guess we'll go ahead and zoom it up one more notot so you can see what I'm typing in here and then from SK learn going to or SK learn cluster we're going to import K means I always forget to capitalize the K and the M when I do this say capital K capital M K means and we'll go and create a um array wcss equals we make it an empty array if you remember from the elbow method from our slide within the sums of squares WSS is defined as the sum of square distance 1:07:16 between each member of the cluster and its centroid so we're looking at that change in differences as far as a squar distance and we're going to run this over a number of K mean values in fact let's go for I in range we'll do 11 of them range 0 of 11 and the first thing we're going to do is we're going to create the actual we'll do all lowercase and so we're going to create this object from the K means that we just imported and the variable that we want to put into this is in clusters and we're going to set that equals to I that's the most important one because we're looking at how increasing the number of clusters changes our answer there are a lot of settings to the K means our guys in the back did a great job just kind of playing with some of them the most common ones that you see 1:08:18 in a lot of stuff is how youit your K means so we have K means plus plus plus this is just a tool let the model itself be smart how it picks it centroids to start with its initial centroids we only want to iterate no more than 300 times we have a Max iteration we put in there we have an the infinit the random State equals zero you really don't need to worry too much about these when you're first learning this as you start digging in deeper you start finding that these are shortcuts that will speed up the process as far as a setup but the big one that we're working with is the in clusters equals I so we're going to literally train our K means 11 times we're going to do this process 11 times and if you're working with uh Big Data you know the first thing you do is you run a small sample of the data so you can test all your stuff on it and you can already see the problem that if I'm going to iterate through a terabyte of data 11 times and then the K means itself is iterating through the data 1:09:18 multiple times that's a heck of a process so you got to be a little careful with this a lot of times though you can find your your elbow using the elbow method find your optimal number on a sample of data especially if you're working with larger data sources so we want to go ahead and take our K means and we're just going to fit it if you're looking at any of the SK learn very common you fit your model and if you remember correctly our variable we're using is the capital x and once we fit this value we go back to the um array we made and we want to go and just to pin that value on the end and it's not the actual fit we're pinning in there it's like when it generates it it generates the value you're looking for is inertia so K means. inertia will pull that specific value out that we need and let's get a visual on this we'll do our PLT plot and what we're plotting here is first the xaxis which is range 01 so that will generate a nice little plot there and the wcss for our y axis 1:10:25 it's always nice to give our plot a title and let's see we'll just give it the elbow method for the title and let's get some labels so let's go ahead and do PLT X label and what we'll do we'll do number of clusters for that and PLT y label and for that we can do oops there we go wcss since that's what we're doing on the plot on there and finally we want to go ahead and display our graph which is simply PLT oops do show there we go and because we have it set to in line it'll appear in line hopefully I didn't make a type error on there and you can see we get a very nice graph you can see a very nice elbow joint there at uh two and again right around three and four and then after that there's not very much now as a data scientist if I was looking at this I would do either three or four and I'd actually try both of them to see what 1:11:26 the um output look like and they've already tried this in the back so we're just going to use three as a setup on here and let's go ahead and see what that looks like when we actually use this to show the different kinds of cars and so let's go ahead and apply the K means to the car's data set and basically we're going to copy the code that we looped through up above where K means equals K means number of clusters and we're just going to set that number of clusters to three since that's what we're going to look for and you could do three and four on this and graph them just to see how they come up differently be kind of curious to look at that but for this we're just going to set it to three go ahead and create our own variable y k means for our answers and we're going to set that equal to whoops I double equal there to K means but we're not going to do a fit we're going to do a fit predict is the setup you want to use and when you're using untrained models you'll see um slightly different usually you see fit and then 1:12:27 you see just the predict but we want to both fit and predict the K means on this and that's fit underscore predict and then our capital x is the data we're working with and before we plot this data we're going to do a little pandas trick we're going to take our x value and we're going to set XS Matrix so we're converting this into a nice rows and columns kind of set up but we want the we're going to have columns equals none so it's just going to be a matrix of data in here and let's go ahead and run that a little warning you'll see These Warnings pop up because things are always being updated so there's like minor changes in the versions and future versions instead of Matrix now that it's more common to set it values instead of doing as Matrix but M Matrix works just fine for right now and you'll want to update that later on but let's go ahead and dive in and plot this and see what that looks like and before we dive into plotting this data I always like to take a look and see see what I am plotting so let's take a look at y k means and I'm 1:13:29 just going to print that out down here and we see we have an array of answers we have 2 1 0 2 one two so it's clustering these different rows of data based on the three different spaces it thinks it's going to be and then let's go ahead and print X and see what we have for x and we'll see that X is an array it's a matrix so we have our different values in the array and what we're going to do it's very hard to plot all the different values in the array so we're only going to be looking at the first two or positions zero and one and if you were doing a full presentation in front of the board meeting you might actually do a little different and and dig a little deeper into the different aspects because this is all the different columns we looked at but we only look at columns one and two for this to make it easy so let's go ahead and clear this data out of here and and let's bring up our plot and we're going to do a scatter plot here so PLT 1:14:30 scatter and this looks a little complicated so let's explain what's going on with this we're going to take the X values and we're only interested in y of K means equals zero the first cluster okay and then we're going to take value zero for the x-axis and then we're going to do the same thing here we're only interested in K means equal Z but we're going to take the second column so we're only looking at the first two columns in our answer or in the data and then the guys in the back played with this a little bit to make it pretty and they discovered that it looks good with as a size equals 100 that's the size of the dots we're going to use red for this one and when they were looking at the data and what came out it was definitely the Toyota on this so we're just going to go ahead and label it Toyota again that's something You' really have to EXP explore in here as far as playing with those numbers and see what looks good we'll go ahead and hit enter in there and I'm just going to 1:15:31 paste in the next two lines which is the next two cars and this is our Nissa and Honda and you'll see with our scatter plot we're now looking at where Yore K means equals 1 and we want the zero column and y k means equals 2 again we're looking at just the first two columns zero and one and each of these rows then corresponds to Nissan and Honda and I'll go ahead and hit enter on there and uh finally let's take a look and put the centroids on there again we're going to do a scatter plot and on the centroids you can just pull that from our c means the uh model we created do cluster centers and we're going to just do um all of them in the first number and all of them and the second number which is 0 one because you always start with zero and one and then they were playing with the size and everything to make it look good we'll do a size of 300 we're going to make the color yellow and we'll label 1:16:32 them so always good to have some good labels centroids and then we do want to do a title PLT title and pop up there PLT title you always make want to make your graphs look pretty we'll call it clusters of car make and one of the features of the plot library is you can add a legend it'll automatically bring in it since we've already labeled the different aspects of the legend with Toyota Nissan and Honda and finally we want to go ahead and show so we can actually see it and remember it's in line uh so if you're using a different editor this not the Jupiter notebook you'll get a popup of this and you should have a nice set of clusters here so we can look at this and we have a clusters of Honda and green Toyota and red Nissan and purple and you can see where they put the centroids to separate them now when we're looking at this we can also plot a lot of other different data on here as far because we only 1:17:33 looked at the first two columns this is just column one and two or 0 one as as you label them in computer scripting but you can see here we have a nice clusters of Carm make and we've able to pull out the data and you can see how just these two columns form very distinct clusters of data so if you were exploring new data you might take a look and say well what makes these different almost going in reverse you start looking at the data and pulling apart the columns to find out why is the first group set up the way it is maybe you're doing loans and you want to go well why is this group not defaulting on their loans and why is the last group defaulting on their loans and why is the middle group 50% defaulting on their bank loans and you start finding ways to manipulate the data and pull out the answers you want so now that you've seen how to use K mean for cluster let's move on to the next topic now let's look into logistic regression the logistic regression algorithm is the simplest classification algorithm used 1:18:35 for binary or multiclassification problems and we can see we have our little girl from Canada whose's into horror books is back that's actually really scary when you think about that with those big eyes in the previous tutorial we learned about linear regression dependent and independent variables so to brush up y = mx + C very basic algebraic function of Y and X the dependent variable is the target class variable we are going to predict the independent variables X1 all the way up to xn are the features or attributes we're going to use to predict the target class we know what a linear regression looks like but using the graph we cannot divide the outcome into categories it's really hard to categorize 1.5 3.6 9.8 uh for example a linear regression graph can tell us that with increase in number of hours studied the marks of a student will increase but it will not tell us whether the student will pass or not in 1:19:36 such cases where we need the output as categorical value we will use logistic regression and for that we're going to use the sigmoid function so you can see here we have our marks 0 to 100 number of hours studied that's going to be what they're comparing it to in this example and we usually form a line that says yal MX + C and when we use the sigmoid function we have P = 1/ 1 + eus y it generates a sigmoid curve and so you can see right here when you take the Ln which is the natural logarithm I always thought it should be n l not Ln that's just the inverse of uh e your e to the minus y and so we do this we get Ln of P / 1 - p = m * x + C that's the sigmoid curve function we're looking for and we can zoom in on the function and you'll see that the function as it deres goes to one or to zero depending on what your x value is and the probability if it's greater than 0.5 the value is 1:20:38 automatically rounded off to one indicating that the student will pass so if they're doing a certain amount of studying they will probably pass then you have a threshold value at the0 five it automatically puts that right in the middle usually and your probability if it's less than 05 the value rented off to zero indicating the student will fail so if they're not studying very hard they're probably going to fail this of course is ignoring the outliers of that one student who's just a natural genius and doesn't need any studying to memorize everything that's not me unfortunately have to study hard to learn new stuff problem statement to classify whether a tumor is malignant or benign and this is actually one of my favorite data sets to play with because it has so many features and when you look at them you really are hard to understand understand you can't just look at them and know the answer so it gives you a chance to kind of dive into what data looks like when you aren't able to understand the specific domain of the data but I also want you to remind you that in the domain of medicine if I told you that my 1:21:38 probability was really good it classified things that say 90% or 95% and I'm classifying whether you're going to have a malignant or a B9 tumor I'm guessing that you're going to go get it tested anyways so you got to remember the domain we're working with so why would you want to do that if you know you're just going to go get a biopsy because you know it's that serious this is like an all or nothing just referencing the domain it's important it might help the doctor know where to look just by understanding what kind of tumor it is so it might help them or Aid them in something they missed from before so let's go ahead and dive into the code and I'll come back to the domain part of it in just a minute so use case and we're going to do our noral Imports here we're importing numpy Pand Seaborn the matplot library and we're going to do matplot library in line since I'm going to switch over to Anaconda so let's go ahead and flip over there and get this started so I've opened up a new window in my anaconda jupyter notebook by the way jupyter notebook uh 1:22:40 you don't have to use Anaconda for the Jupiter notebook I just love the interface and all the tools that Anaconda brings so we got our import nump as in P for our numpy number array we have our Panda PD we're going to bring in Seaborn help us with our graphs as SNS so many really nice Tools in both caborn and matplot library and we'll do our matplot library. pyplot as PLT and then of course we want to let it know to do it in line and let's go and just run that so it's all set up and we're just going to call our data data not creative today uh equals PD and this happens to be in a CSV file so we'll use the pd. read CSV and I happen to named the file I renamed it data for 1:23:53 p2.png which test was done the diagnosis M for malignant B for B9 so there's two different options on there and that's what we're going to try to predict is the m andb and test it and then we have like the radius mean or average the texture average perimeter mean area mean smoothness I don't know about you but unless you're a doctor in the field most of the stuff I mean you can guess what concave means just by the term concave but I really wouldn't know what that means in the measurements they're taking so they have all kinds of stuff like how smooth it is uh the Symmetry and these are all float values we just page through them real quick and you'll see there's I believe 36 if I remember correctly in this one so there's a lot of different values they take and all these measurements they take when they go in there and they take a look at the different growth the tumorous growth so back in our data and I put this in the same folder as a code so I saved this code in that folder obviously if you have it in a different location you want to put the full path 1:24:54 in there and we'll just do uh Panda's first five lines of data with the data. head and we run that we can see that we have pretty much what we just looked at we have an ID we have a diagnosis if we go all the way across you'll see all the different columns coming across displayed nicely for our data and while we're exploring the data our caborn which we referenced as SNS makes it very easy to go in here and do a joint plot you'll notice the very similar to because it is sitting on top of the um plot Library so the joint plot does a lot of work for us and we're just going to look at the first two columns that we're interested in the radius mean and the texture mean we'll just look at those two columns and data equals data so that tells it which two columns we're plotting and that we're going to use the data that we pulled in let's just run that and it generates a really nice graph on here 1:25:55 and there's all kinds of cool things on this graph to look at I mean we have the texture mean and the radius mean obviously the axes you can also see and one of the cool things on here is you can also see the histogram they show that for the radius mean whereas the most common radius mean come up and where the most common texture is when we're looking at the tech the on each growth its average texture and on each radius its average uh radius on there gets a little confusing because we're talking about the individual objects average and then we can also look over here and see the the histogram showing us the median or how common each measurement is and that's only two columns so let's dig a little deeper into Seaborn they also have a heat map and if you're not familiar with heat Maps a heat map just means it's in color that's all that means heat map I guess the original ones were plotting heat density on something and so ever since then it's just called a heat map and we're going to take our data and get our 1:26:55 corresponding numbers to put that into the heat map and that's simply data. C RR for that that's a pandas expression remember we're working in a pandas data frame so that's one of the Cool Tools in pandas for our data and this is pull that information into a heat map and see what that looks like and you'll see that we're now looking at all the different features we have our ID we have our texture we have our area our compactness concave points and if you looked at on the middle of this chart diagonal going from the upper left to bottom right it's all white that's because when you compare texture to texture they're identical so they're 100% or in this case perfect one in their correspondence and you'll see that when you look at say area or right below it it has almost a black on there when you compare it to texture so these have almost no corresponding data They Don't Really form a linear graph or something that you can look at and say the hell can Ed they are they're very scattered data this is really just a really nice 1:27:58 graph to get a quick look at your data doesn't so much change what you do but it changes verifying so when you get an answer or something like that or you start looking at some of these individual pieces you might go hey that doesn't match according to showing our heat map this should not correlate with each other and if it is you're going to have to start asking well why what's going on what else is coming in there but it does show some really cool information on here mean we can see from the ID there's no real one feature that just says if you go across the top line that lights up there's no one feature that says hey if the area is a certain size then it's going to be B9 or malignant it says there's some that sort of add up and that's a big hint in the data that we're trying to ID this whether it's malignant or B9 that's a big hint to us as data scientist to go okay we can't solve this with any one feature it's going to be something that includes all the features or many of the different features to come up with the solution for it and while we're 1:29:00 exploring the data let's explore one more area and let's look at data. is null we want to check for null values in our data and if you remember from earlier in this tutorial we did it a little differently where we added stuff up and summed them up you can actually with pandas do it really quickly data. isnull and Summit and it's going to go across all the columns so when I run this you're going to see all the columns come up with no null data so we've just just to reash these last few steps we've done a lot of exploration we have looked at the first two columns and seeing how they plot with the caborn with a joint plot which shows both the histogram and the data plotted on the XY coordinates and obviously you can do that more in detail with different columns and and see how they plot together and then we took and did the Seaborn heat map the SNS do heat 1:30:00 map of the data and you can see right here where it did a nice job showing us some bright spots where stuff correlates with each other and forms a very nice combination or points of scattering points and you can also see areas that don't and then finally we went ahead and checked the data is the data null value do we have any missing data in there very important step because it'll crash later on if you forget to do this step it will remind you when you get that nice error code that says null values okay so not a big deal if you miss it but it it's no fun having to go back when you're you're in a huge process and you've missed this step and now you're 10 steps later and you got to go remember where you were pulling the data in so we need to go ahead and pull out our X and our y so we just put that down here and we'll set the x equal to and there's a lot of different options here certainly we could do x equals all the columns except for the first two because if you remember the first two is the ID 1:31:00 and the diagnosis so that certainly would be an option but what we're going to do is we're actually going to focus on the worst the worst radius the worst texture parameter area smoothness compactness and so on one of the reasons to start dividing your data up when you're looking at this information is sometimes the data will be the same data coming in so if I have two measurements coming into my model it might overweigh them it might overpower the other measurements because it's measuring it's basically taking that information in twice that's a little bit past the scope of this tutorial I want you to take away from this though is that we are dividing the data up into pieces and our team in the back went ahead and said hey let's just look at the worst so I'm going to create a an array and you'll see this array radius worst texture worst perimeter worst we've just taken the worst of the worst first and I'm just going to put that in my X so this x is still a pandas data frame but it's just those columns and our y if you remember 1:32:02 correctly it's going to be oops hold on one second it's not X it's data there we go so x equals data and then it's a list of the different columns the worst of the worst and if we're going to take that then we have to have our answer for our Y for the stuff we know and if you remember correctly we're just going to be looking at the diagnosis that's all we care about is what is it diagnosed is it Bine or malignant and since it's a single column we can just do diagnosis oh I forgot to put the brackets the there we go okay so it's just diagnosis on there and we can also real quickly do like x. head if you want to see what that looks like and Y do head and run this and you'll see um it only does the last one I forgot about that if you don't do print you can see that the the Y do head is just in mmm cuz the first ones are all malignant and if I run this the x. head is just the first five values of radius worst texture worst parameter 1:33:02 worst area worst and so on I'll go ahead and take that out so moving down to the next step we've built our two data sets our answer and then the features we want to look at in data science it's very important to test your model so we do that by splitting the data and from sklearn model selection we're going to import train test split so we're going to split it into two groups there are so many ways to do this I noticed in one of the more modern ways they actually split it into three groups and then you model each group and test it against the other groups so you have all kinds and there's reasons for that which is past the scope of this and for this particular example isn't necessary for this we're just going to split it into two groups one to train our data and one to test our data and the sklearn uh. model selection we have train test split you could write your own quick 1:34:02 code to do this where you just randomly divide the data up into two groups but they do it for us nicely and we actually can almost we can actually do it in one statement with this where we're going to generate four variables capital x train capital X test so we have our training data we're going to use to fit the model and then we need something to test it and then we have have our y train so we're going to train the answer and then we have our test so this is the stuff we want to see how good it did on our model and we'll go ahead and take our train test split that we just imported and we're going to do X and our y our two different data that's going in for our split and then the guys in the back came up and wanted us to go ahead and use a test size equals three that's testore size random State it's always nice to kind of switch a random State around but not that important what this means is that the test size is we're going to take 30% of the data and we're going to put that into our test variables our y test and our X test and 1:35:04 we're going to do 70% into the X train and the Y train so we're going to use 70% of the data to trainer model and 30% to test it let's go ahead and run that and load those up so now we have all our stuff split up and all our data ready to go now we get to the actual Logistics part we're actually going to do our create our model model so let's go ahead and bring that in from sklearn we're going to bring in our linear model and we're going to import logistic regression that's the actual model we're using and this we'll call it log model there we go model and let's just set this equal to our logistic regression that we just imported so now we have a variable log model set to that class for us to use and with most the uh models in the SK learn we just need to go ahead head and fix it fit do a fit on there and we use our X train that we separated out with our y train and let's go ahead and run this so once we've run 1:36:04 this we'll have a model that fits this data that 70% of our training data uh and of course it prints this out that tells us all the different variables that you can set on there there's a lot of different choices you can make but for word do we're just going to let all the defaults sit we don't really need to mess with those on this particular example and there's nothing in here that really stands out as super important until you start fine-tuning it but for what we're doing the basics will work just fine and then let's we need to go ahead and test out our model is it working so let's create a variable y predict and this is going to be equal to our log model and we want to do a predict again very standard format for the sklearn library is taking your model and doing a predict on it and we're going to test y predict against the Y test so we want to know what the model thinks it's going to be that's what our y predict is and with that we want the capital x x test so we have our train set and our test set and now we're 1:37:05 going to do our y predict and let's go ahead and run that and if we uh print y predict let me go ahead and run that you'll see it comes up and it PR a prints a nice array of uh B and M for B9 and malignant for all the different test data we put in there so it does pretty good we're not sure exactly how good it does but we can see that it actually works and it's functional was very easy to create you'll always discover with our data science that as you explore this you spend a significant amount of time prepping your data and making sure your data coming in is good uh there's a saying good data in good answers out bad data in bad answers out that's only half the thing that's only half of it selecting your models becomes the next part as far as how good your models are and then of course fine-tuning it depending on what model you're using so 1:38:06 we come in here we want to know how good this came out so we have our y predict here log model. predict X test so for deciding how good our model is we're going to go from the SK learn. metrics we're going to import classification report and that just reports how good our model is doing and then we're going to feed it the model data let's just print this out and we'll take our classification report and we're going to put into there our test our actual data so this is what we actually know is true and our prediction what our model predicted for that data on the test side and let's run that and see what that does so we pull that up you'll see that we have um a Precision for B9 and malignant B&M and we have a Precision of 93 and 91 a total of 92 so it's kind of the average between these two of 92 1:39:07 there's all kinds of different information on here your F1 score your recall your support coming through on this and for this I'll go ahead and just flip back to our slides that they put together for describing it and so here we're going to look at the Precision using the classification report and you see this is the same print out I had up above some of the numbers might be different because it does randomly pick out which data we're using so this model is able to predict the type of tumor with 91% accuracy so we look back here that's you will see where we have uh B9 and mland it actually is 92 coming up here but we're looking about a 92 91% precision and remember I reminded you about domain so we're talking about the domain of a medical domain with a very catastrophic outcome you know at 91 or 92% Precision you're still going to go in there and have somebody do a biopsy on it very different than if you're investing money and there's a 92% chance you're going to earn 10% and 8% chance 1:40:08 you're going to lose 8% you're probably going to bet the money because at that odds it's pretty good that you'll make some money and in the long run you do that enough you definitely will make money and also with this domain I've actually seen them use this to identify different forms of cancer that's one of the things that they're starting to use these models for because then it helps the doctor know what to investigate so that wraps up this section we're finally we're going to go in there and let's discuss the answer to the quiz asked in machine learning tutorial part one can you tell what's happening in the following cases grouping documents into different categories based on the topic and content of each document this is an example of clustering where K means clustering can be used to group the documents by topics using bag of words approach so if You' gotten in there that you're looking for clustering and hopefully you had at least one or two examples like K means that are used for clustering different things then give yourself a two thumbs up B identifying handwritten digits and images correctly 1:41:09 this is an example of classification the traditional approach to solving this would be to extract digit dependent features like curvature of different digits Etc and then use a classifier like svm to distinguish between images again if you got the fact that it's a classification example give yourself a thumb up and if you're able to go hey let's use svm or another model for this give yourself those two thumbs up on it C behavior of a website indicating that the site is not working as designed this is an example of anomaly detection in this case the algorithm learns what is normal and what is not normal usually by observing the logs of the website give yourself a thumbs up if you got that one and just for a bonus can you think of another another example of anomaly detection one of the ones I use for my own business is detecting anomalies in stock markets stock markets are very ficked and they behave very ertical so finding those erratic areas and then finding ways to track down why they're erratic was something released in social 1:42:10 media was something released you can see where're knowing where that anomaly is can help you to figure out what the answer is to it in another area D predicting salary of an individual based on his or her years of experience experience this is an example of regression this problem can be mathematically defined as a function between independent years of experience and dependent variables salary of an individual and if you guess that this was a regression model give yourself a thumbs up and if you're able to remember that it was between independent and dependent variables and that terms give yourself two thumbs up the principal component uh analysis we're going to cover dimensionality reduction principal component analysis what is it important PCA terminologies and you'll see it abbreviated uh normally as PCA principal component analysis PCA properties PCA example and then we'll pull up some python code in our jupyter notebook and have some Hands-On demo on the PCA and how it's used dimensionality 1:43:12 reduction dimensionality reduction refers to the technique that reduces the number of input variables in a data set and so you can see on the table on the right shows the orders made at an automobile parts retailer the retailer sells different automobile parts from different companies and you can see we have company uh bpex isomax and they have the item the tire the axle an order ID a price number and a quantity in order to predict the future cells we find out that using correlation analysis that we just need three attributes therefore we have reduced the number of attributes from 5 to three and clearly we don't really care about the part number I don't think the part number would have an effect on how many tires are bought um and even the store who's buying them probably does not have an effect on that in this case that's what they've actually done is remove those and we just have the item the tire the price and the quantity one of the things you should be taking away from this is 1:44:12 in the scheme of things we are in the descriptive phase we're describing the data and we're pre-processing the data what can we do to clean it up why dimensionality reduction well number one less dimensions for a given data set means less computation or training time that can be really important if you're trying a number of different models uh and you're rerunning them over and over again and even if you have seven gigabytes of data that can start taking days to go through all those different models so this is huge this is probably the hugest part as far as reducing um our data set redundancy is removed after removing similar entries from the data set again pre-processing some of our models like a neural network if you put in two of the same data it might give them a higher weight than they would if it was just once so we want to get rid of that redundancy it also increases the processing time if you have multiple uh data coming in space required to store 1:45:14 the data is reduced so if we're committing this into a big data uh pool we might not send the company that bought it why would we want to store two whole extra columns when we add it into that pool of data makes the data easy for plotting in 2D and 3D plots this is my favorite part very important you're in your shareholder meeting you want to be able to give them a really good um clear and simplified version you want to reduce it down to something people can take in it helps to find out the most significant features and skip the rest which also comes in in post scribing uh leads to better human interpretation that kind of goes with number four what makes data easy for plotting you have a better interpretation when we're looking at it principal component analysis so what is it uh principal component analysis is a technique for reducing the dimensionality of data sets increasing interpretability but at the same time minimizing information loss so we take 1:46:15 some very complex data set with lots of variables we run it through the PCA we reduce the variables we end up with a reduced variable setup this is very confusing to look at because if you look at the end result we have the different colors all lined up so what we're going to take a look at is let's say we have a picture here uh let's say you are asked to take a picture of some Toddlers and you were deciding which angle would be the best to take the picture from so if we come up here we look at this we say okay this is you know one angle uh we get the back of a lot of heads not many faces uh so we'll do it from here we might get the one person up front smiling a lot of the people in the class are missing so we have a huge amount off to the right of blank space Maybe from up here again we have the back of someone's head and uh it turns out that the best angle to click the picture from might be this bottom left angle you look at it and you say hey that makes sense it's a a good um configuration of all the people in the picture now when we're 1:47:17 talking about data it's not you really can't do it by what you think is going to be the best need to have some kind of mathematical formula so it's consistent and so it makes sense in the back end one of the projects I worked on many years ago uh has something similar to the iris and if you've ever done the iris data set it's probably one of the most common ones out there where they have the flower and they're measuring the stammen uh in the petals and they have width and they have length of the petal instead of putting through the width and the length of the petal we could just as easily do the um width to length ratio we can divide the width by the length and you get a single number where you had two that's a kind of idea that's going on into this in pre-processing and looking at what we can do to bring the data down a very simplified example on my uh Iris pedal example when we look at the similarity in PCA we find the best picture or projection of the data points and so we 1:48:20 look down at from one angle we've drawn a line down there uh we can see these data points based on in this case just two variables now keep in mind we're usually talking about 36 40 variables um almost all of your business models usually have about 26 to 27 different variables they're looking at uh same thing with like a bank loan model we're talking 26 to 36 different variables they're looking at that are going in so we want to do is we want to find the best view in this case we're just looking at the XY we look down at it and we have our second um idea PC2 and again we're looking at the XI this XY this time from a different direction here for our ease we can consider that we get two principal components namely pc1 and PC2 comparing both the principal components we find the data points are sufficiently spaced in pc1 so if you look at what we got here we have uh pc1 you can see along the 1:49:21 line how the data point Point are spaced versus the spacing in PC2 and that's what they're coming up with what is going to give us the best look for these data points when we combine them and we're looking at them from just a single angle whereas in PC2 they are less spaced which makes the observation and further calculations much more difficult therefore we accept the pc1 and not the PC2 as the data points are more spaced now obviously the backend calculations are a little bit more complicated when we get into the math of how they decide side what is more valuable this gives you an idea though that when we're talking about this we're talking about the perspective uh which would help in understanding how PCA analysis works we want to go ahead and do is dive into the important terminologies under a PCA and important terminologies are views the perspective through which data points are observed and so you'll hear that if someone's talking about a PCA presentation and they're not taking the 1:50:23 time to reduce it to something that the average person shareholders can understand you might hear and refer to it as the different views what view are we taking Dimension number of columns in a data set are called the dimensions of that data set and we talked about you'll hear features Dimensions um I was talking about features there's usually when you're running a business you're talking 25 26 27 different features minimal and then you have the principal component new variables that are constructed as linear combinations or mixtur of the initial variables principal component is very important it's a combination if you remember my flower example it would be the width over the length of the petal as opposed to putting both width and length in you just put in the um ratio instead which is a single number versus two separate numbers projections the perpendicular distance between the principal component and the data points and that goes to that line we had earlier it's that right angle line of where the Point how those points fall 1:51:24 onto the line important properties important properties number of principal components is always less than or equal to the number of attributes that just makes Common Sense uh you're not going to do 10 principal properties with only three features you're trying to reduce them so that's just kind of goofy but it is important to remember that people will throw weird code out there and just randomly do stuff with instead of really thinking it through principal components are or agonal and this is what we're talking about that right angle from the line we when we do pc1 we're looking at how those points fall onto that line uh same thing with PC2 we want to make sure that pc1 does not equal PC2 we don't want to have the same two principal points uh when we do two points the priority of principal components decreases as their numbers increase the this is important to understand if you're going to 1:52:26 create uh one principal component everything is summarized into that one component as we go to two components the priority um how much it holds value decreases as we go down so if you have five different points each one of those points is going to have less value than just the one point which has everything summarized in it how PCA works I said there was more in the back end when we talk about the math this is what we're talking about is how does it actually work so now we have understanding that you're you're looking at a perspective uh now we want to see how that math side Works PCA performs the following operations in order to evaluate the principal components for a given data set first we start with the standardization then we have a covariance matrix computation and we use that to generate our ige vectors and ige values which is the 1:53:28 feature vector and if you're remember the IG Vector is like a translation for um moving the data from xal 1 to xal 2 or whatever altering it and the igene value is the final value that we generate when we talk about standardization the main aim of this step is to standardize the range of the attributes so that each one of them lie within similar bound boundaries this process involves removal of the mean from the variable values and scaling the data with respect to the standard deviation and you can see here we have Z equals the variable values minus the mean over the standard deviation The covariance Matrix computation covariance Matrix is used to express the correlation between any two or more attributes in multi-dimensional data set the covariance Matrix has the entry as the variance and the co-variance of the attribute values the variance is denoted by VAR and the co variance is denoted by Cove on the right 1:54:31 we can see the co-variance Matrix for two attributes and their values when we do a Hands-On look at the code we'll do a display of this so you can see what we're talking about and what that looks like for now you can just notice that this is a matrix that we're generating with the variance and then the co-variance of x to Y on the right side we can see the co-variance table for more than two attributes in a multi-dimensional data set this is what I was talking about we usually are looking at uh not just one feature or two features we're usually looking at 25 30 features going on and so if we do a setup like this we should see all those different features as the different variables covariance Matrix tells us how the two or more variables are related positive covariant indicate that the value of one variable is directly proportional to the other variable negative covariant indicate that the value of one variable is inversely proportional to the other variable that 1:55:31 is always important to note whenever we're doing any of these matrixes that we're going to be looking at that positive and negative whether it's inverted or not and then we have the ige values and the igene vectors igene values and IG vectors are the mathematical value that are extracted from the co-variance table they are responsible for the generation of a new set of variables from the old set of variables which further lead to the construction of the principal components IG vectors do not change directions after linear transformation ige values are the scalars or the magnitude of the ige vectors and again this is just change transforming that data so we're going to change uh the vector B to the B Prime as denoted on the chart and so when we have like multiple variables how do we calculate that new variable and then we have feature vectors feature vectors is simply a matrix that has IG vectors of the components that we decide 1:56:31 to keep as the columns here we decide whether we must keep or discard the less significant principal components that we have generated in the above steps this becomes really important as we start looking at uh the back end of this and we'll do this in the demo uh but one of the more important steps to understand and so we have the PCA example consider Matrix x with n rows or observations and K columns or variables now for this Matrix we would construct a variable space with as many dimensions as the variable but for our Simplicity let's consider this three dimensions for now now each observation row of the Matrix X is placed in the K dimensional variable space such that the rows in the data table form a swarm of points in this space now we find the mean of all the observations and then place it along the data points on the plot the first principal component is a 1:57:33 line that best accounts for the shape of the point swarm it represents a maximum variance Direction in the data each observation may be projected onto this line in order to get a coordinate value along the pc1 this value is known as a score usually only one principal component is insufficient to model the systematic variation for a data set thus a second principal axis is created the second principal component is oriented such that it reflects the second largest source of variation in the data while being orthagonal to pc1 PC2 also passes through the average Point let's go ahead and pull this up and just see what that means uh inside our python scripting I'm going to use the anacon a navigator and I will be in Python 3.6 for this example I believe there's even like a 3.9 out I tend to stay in 3.6 because a lot of the models I use 1:58:35 especially with the neural networks are stable in 36 and then we open up our uh Jupiter I'm in Chrome and we go ahead and create a new Python 3 and for ease of use uh our team in the back was nice enough to put this together for me and we'll go a and start with the libraries the first thing I like to do whenever I'm looking at any new setup uh well you know what Let's do let's do the libraries first we're going to do our basic libraries which is matplot Library uh the PLT from the matplot library pandas our data frame uh PD numpy or numbers array NP caborn for graphing SNS that goes with the plot that actually sits on Matt plot Library so the Seaborn sits on there and then we have our Amber sign because we're in Jupiter notebook matplot library in line the newer version actually doesn't require that um but I put it in there either anyway just because I'm so used 1:59:37 to it and then we want to go ahead and take a look at the data and in this case we're going to pull in uh certainly you can have lots of fun with different data but we're going to use the cancer data set uh one of the reasons a cancer data set is is it has like 36 35 different features so it's kind of fun to use that as our base for this and we'll go ahead and run this and look at our keys and the first thing we notice in our keys for the cancer data set uh we have our data we have our Target our frame Target names description feature names and file name so what we're looking for in all this is um well let's take a look at the description let's go in here and pull up the description on here I'm not going to spend a huge amount of time on the description um because this is we don't want to get into a medical domain we want to focus on our PCA setup uh what's important is 2:00:39 you start looking at what the different attributes are what they mean um if you were in the medical field you'd want to note all these different things whether what they're measuring where it's coming from you can actually see the actual different um measurements are taking no missing attributes we page all the way to the bottom and you're going to have your data in this case our Target and if you dig deep enough to the Target uh let's actually do this let's go ahead and print Target names real quick here I always like to to just take a look and see what's on the other end of this uh uh Target names run that so the target name is is it malignant or is it B9 um so in other words is this uh dangerous growth or is it something we don't have to worry 2:01:40 about that's the bottom line with the cancer in this case and then we can go ahead and load our data and uh you know what let me go up a just a notch here for easy reading it's hard to get that just right I guess I'll be have to do uh so let's go ahead and look at our data uh our our we're going to use our pandas and we're going to go ahead and uh do our data frame it's going to equal cancer data columns equals cancer feature equals feature names so remember up here we already loaded the the U names up of our of the features in there what is going to come out of this let me just see if I can get to that it's at the top of Target names um that's just this list of names here in the setup and we can go ahead and run this code and it'll print the head and you can see here we have the mean radius the mean texture mean perimeter I don't know about you this is a wonderful data set 2:02:40 if you're playing with it because like many of the data that most of the data that comes in half the time we don't even know what we're looking at uh we're just handed a bunch of stuff as a data scientist going what the heck is this and so this is a good place to start because this has a number of different features in there and we have no idea what these feature means or where they come from we want to just look at the data and figure that out and now we actually getting into the PCA side of it as we've noticed before it is difficult to visualize High dimensional data we can use PCA to find the first two principal components and visualize the data this new two-dimensional space with a single scatter plot uh before we do this we need to go ahead and scale our data now I haven't run this to see if you really have to scale the data on this um but as just a general runtime I almost do that as the first step of any modeling even if it's pre-modeling as we're doing here um in 2:03:42 neural networks that is so important with PCA visualization it's already going to scale it when we do the means and deviation inside the PCA uh but just in case it's always good to scale it and then we're going to take our PCA with the pyit learn uses very similar process to other pre-processing functions that come with pyit learn we instantiate a PCA object find the princial components using the fit method then apply the rotation and dimensionality reduction by calling transform we can also specify how many components we want to keep when creating the PCA object and so the code for this oops getting a little bit ahead let me go and run this code uh so the code for this is from sklearn decomposition import PCA PCA equals PCA in components equals 2 and that's really 2:04:44 important to note that um because we're only going to want to look at two components I would never go over four components uh especially if you're going to demo this with somebody else if you're showing this to the shareholders the whole idea is to reduce it to something people can see uh and then the PCA fit we're going to uh is going to take the scale data that we generated up here and then you can see we've created our PCA model with in components equals 2 now whenever I use a new tool I like to go in there and actually see what I'm using so let's go to the S kit uh web page for the PCA and you can see in here here's our call statement it describes what all the different uh setups you have on there probably the biggest one to look at would be um well the biggest one is your components how many components do you want uh which you have to put in there pretty much and then you also might look at the SVD solver it's on 2:05:45 auto right now but you can override that and do different things with it it does a pretty good job as it is and if we go down all the way down to um here we go to our methods if you notice we have fit we have fit transform nowhere in here is predict because this is not used for prediction uh it's used to look at the data again we're in the describe setup we're fitting the data we're taking a look look at it uh we've already looked at our minimum maximum we've already looked at what's in each quarter we've done a full description of the data this is part of describing the data um that's the biggest thing I take away when I come zooming in here and of course they have examples of it down here if you forget um and the biggest one of course is the number of components and then uh I mean the rest you can play with um the 2:06:47 actual solver whether you're doing a full randomize there's different things it just pretty good on the auto and now we can transform this data to its first two principal components and so we have our um X PCA we're going to set that equal to PCA transform scaled data uh so there we go there's our first transformation and let's just go ahead and print the scaled data shape and the xpc data shape and the reason we want to do this is just to show us uh what's going on here we've taken 30 features I think I said 36 or something like that but it's 30 and we've compressed it down to two features and we decided we wanted two features and that's where this comes from uh we still have 569 data sets I mean data rows not data sets we still have 569 rows of data but instead of 2:07:48 computing 30 features we're now only doing our model on two features so let's go ahead and uh uh plot these and take a look and see what's going on and uh we're just going to use our PLT figure we're going to set the figure size on here here's our scatter plot um X PCA xcore PCA of uh of one these are two different perceptions we're using uh and then you'll see right here C for color cancer equals Target and so remember we have zero we have one and if I remember correctly zero was malignant one was B9 uh so everything in the zero column is going to be one color and the other color is going to be one and then we're going to use the plasma map just kind of telling you what color it is add some labels first principal component second principal component we'll go and run 2:08:50 this and you you can see here instead of having a chart one of those heat maps with 30 different columns in it we can look at this and say hey this one actually did a pretty good job of separating the data and a couple things when I'm looking at this that I notice is first we have a very clear area where it's clumped together um where it's going to be B9 and we have a huge area it's still together more spread out where it's going to be uh malignant or I think I had that backwards and then in the middle because we're dealing with something in this particular case cancer we would try to separate I would be exploring how to separate this middle group out in other words there's an area where everything overlaps and we're not going to have a clear result on it uh just because those are the people you want to go in there and have extra tests or treat it differently versus going in 2:09:50 and saying just cutting into the can into the cancer so the body absorbs it and it dissipates versus uh actively going in there removing it testing it going through chemo and all the different things that's a big difference you know as far as what's going to happen here in that middle line where they overlap is going to be huge that's domain specific uh going back to the data we can see here uh clearly by using these two components we can easily separate these two classes so the next step is what does that mean interpreting the components unfortunately with this great power of dimensionality reduction comes a cost of not being able to easily understand what these components represent I don't know what principle component one looks represents or second principle the components correspond to combinations of original features the components themselves are stored as an attribute of the filtered PCA object and so when we t look at that we can go ahead and do uh look at the PCA 2:10:52 components this is in our model we built we've trained it we can run that and you can see here's the actual components uh it's the two components have each have their own array and within the array you can see the um what the scores they're using and these actually give weight to what features are doing what so in this numpy Matrix array each row represents a principal component and each column relates back to the original features what's really neat about this is we can now go in reverse and drop this onto a heat map and start seeing uh what this means and so let me go ahead and just put this down o I already got it down here uh we go ahead and put this in here we're going to use our um DF comp data frame and we do our PCA components and I want you to notice how easy this is uh we're going to set our columns equal to cancer feat feat names 2:11:52 that just makes it really easy and we're dumping it into a data frame what's neat about a data frame is when we get to caborn it will pull that data frame apart and U and set it up for us where we want and so we're just going to do the CN the Seaborn heat map of our data frame uh composition and we'll use a plasma coloring and it creates a nice little color graph here you can see we have the mean radius and all the different features along the bottom on the right uh we have a scale so we can see we have the dark colors all the way to the really light colors which are what's really shining there this is like the primary stuff we want to look at so this heat map and the color bar basically represent the correlation between the various features and the principal component itself so you know very powerful map to look at and then you can go in here and we might notice that the mean radius 2:12:53 look how how on the bottom of the map it is um on some of this uh so we have some interesting correlations here that change the variations on that and what means what this is more when you get to uh uh 2:13:21 data understanding linear regression bias and variance what is overfitting what is underfitting and those are like the biggest things right now in data science is overfitting and underfitting what does that mean and what is regularization and then we'll do a quick Hands-On demo to take a look at this so fitting the data let's start with fitting the data and we talk about uh what is data fitting it's a process of plotting a series of data points and drawing the best fit line to understand the relationship between the variables and this is what we called Data fitting and you can see here we have a couple of lines we've drawn on this graph we're going to go in a little deeper on there uh so we might have in this case just a two Dimensions we have an efficiency of the car and we have the distance traveled in 1,000 kilomet and so what is data fitting well it's a linear relationship and a linear relationship very specifically linear means line line uh the line used to 2:14:23 represent the relationship is a straight line that passes through the data points and the variables have linear relationship linear regression so let's start with uh how linear regression Works a linear regression uh finds the line that best fits the data point and gives a relationship between the two variables and so you can see here we have the efficiency of the car um versus the distance traveled and you can see this nice straight line drawn through there and when you talk about multiple variables all you're doing is putting this instead of a line it now becomes a plane it gets a little more complicated with multiple variables but they all come down to this linear kind of uh drawing a line through your data and finding what fits the the data the best and so we can consider an example uh let's say that we want to find the relationship between the temperature outside versus the celles of ice cream and so we start looking at that we're looking at the how many ice cream cones we're selling or how much money we sold in ice cream and we're looking at how 2:15:26 warm it is outside which would hopefully draw a lot of people into the ice cream store and suppose we have two lines we're going to draw L1 and L2 and we're going to kind of guess which one we think is the best fit and which claim to describe the relationship between the variables and so first we find the square of the distance between the line L1 and each data point and add them all in find the mean distance and I want you to think about that when we um uh Square something if it's a negative or positive number it no longer matters because a minus 2^ 2ar is 4 2^ 2 is 4 so we're removing what side of the line it's on and we're just looking for the error in this case the mean distance of each of the little dotted lines you see here this way of calculating the square of the distance adding them and then taking the mean is called mean square error or loss function and we talk about loss how far off are we that's what 2:16:27 we're really talking about what did we miss uh and we have a positive distance and a negative distance and of course when we Square it it is neither just becomes a positive error and so we take the mean square error and a lot of times you'll see it referred to as MSE uh if I look in a code and I'm going through my python code and I see MSE I know that's a mean squared error and take all the dotted lines and we calculate this error we add them all together and then we average it or find the means and in this case they ran a demo on this and it was uh 1,1 127.23 uh for our L1 line now we find the loss function for line L2 in a similar fashion and we get the mean square error to be 6,397 and it's computed the same way so maybe you put this line just way outside the D range and this is the error you get by analyzing our results we find 2:17:27 that the loss function or the mean square error is less for L1 than L2 hence L1 is the best fit line uh this process describes a lot of machine learning processes we're going to keep guessing and get as close as we can to find the right answer we have to have some way to inv to um calculate this and figure out which one's the best and the mean square error is one of the better fits to doing for doing this and most commonly used we really want to talk about bias and variance very important terms to know in machine learning and with linear regression so bias bias occurs when an algorithm has limited flexibility to learn from data variance defines the algorithm's sensitivity to specifics sets of data let's start with bias and variance you can see here we have um the two 2:18:27 different setups uh bias you can think is very generalized where variance is very specific uh and so we talk about bias such models pay very little attention to the training data and oversimplify the model therefore the validation error or prediction error and training error follow similar Trends and uh with bias if you oversimplify it so much you're going to miss your um local uh if if you have like a really good fit you're going to miss it you're you're going to just kind of guess what the average is and that's what your answer is going to be with variance a model with a high variance pays a lot of attention to training data and does not generalize therefore the validation error or prediction error are far apart from each other such models always lead to a high error on training and test data as a bias does where variance such models usually perform very well on training data but have high error rates on test 2:19:29 data and I want you to think about this um when we're talking about a bias uh the error is going to be high both when you're training it and you're testing it why because we're just kind of getting an average we're not really fitting it close with variance we're fitting it so close that the test data does really good it's going to nail it every time uh if you're doing categorical testing that's a car that's a truck that's a bicycle um but with variant suddenly a truck has to have certain features and it might have to be red uh because you had so many red pictures so if it has if it's an 18-wheeler it has to be red if it's uh blue then it has to be a bicycle that's the kind of variance we're talking about where it picks up on something and it it cannot get the right answer unless it gets a very specific data and we see that so that as you're testing it your models and you programmed it you got to look for how I 2:20:30 trained it what is coming out and if it's not if it's not looking good on either bias or or if it's not looking good on the training or on the test data than your bias uh than your bias in your data if it really looks good on the training data then that's going to be your variance you've overfitted the data and those are very important things to know when you are building your models uh in regression of any kind or any kind of uh setup for predicting so in dark games if all the data fall a particular pointer this can be considered as a biased throw and the player aims for the particular score for variance if all the darts fall on different pointers and no two darts fall on the same pointer then this can be considered as a varied throw and the player aims for various scores uh again the bias sums everything up in one point kind of averages it together where the variance really looks for the individual uh predictions coming out so let's go ahead and talk about 2:21:32 overfitting uh when we talk about overfitting it's a scenario where the machine learning model tries to learn from the details along with the noise and the data tries to fit each data point on the curve uh you can see that um if you plug in your coordinates you're just going to get the whatever it's fitted every point on the data stream there's no average there's no two points that might have that you me know why might have two different answers because the if the wind blows a certain way um and your efficiency of your car maybe you have a headwind so your car might alter how efficient it is as it goes and so there's going to be this variance on here and this says no you can't have any variance what you know that this is it's going to be exactly this it can't be any you can't be the same speed or the same car and have a slightly different efficiency so as the model has very less flexibility it fails to predict new data points and thus the model rejects every 2:22:32 new data point during the prediction uh so you'll get like a really high error on here and so uh reasons for overfitting uh data used for training is not cleaned and contains noise garbage values in it you can spend so much time cleaning your data and it's so important it's so important that if you have if you have some kind of something wrong with the data coming in it needs to be addressed whether it's a source of the data maybe they use in medical different measuring tools uh so you now have to adjust for data that came in from hospital a versus Hospital b or even off of machine a and machine B that's testing something and those those numbers are coming in wrong the model has a high variance uh again when is a good example I was talking about that with the car you may have 100 tests but because the wind's blowing it's all over the place uh size of training data used is not enough so a small amount of data is 2:23:33 going to also cause this problem you only have a few points and you try to plot everything the model is too complex uh this comes up a lot we put too many pieces together and how they interact can't even be tracked um and so you have to go back break break it up and find out actually what correlates and what doesn't so what is underfitting a scenario where machine learning models can neither learn the relationship between the data points nor predict or classify a new data point and you can see here we have our efficiency of our car and our line drawn and it's just going to be way off for both the training and the predicting data as the model doesn't fully learn the pattern pns it accepts every new data point during the prediction so instead of looking for a general pattern uh we just kind of accept everything data used for training is not cleaned and contains noise garbage and 2:24:34 values again underfitting and overfitting same issue you got to clean your data the model has a high bias uh we've seen this in all kinds of things from uh the mo the most common is the driving cars to facial identification or whatever it is the model itself when they build it might have a bias towards one thing and this would be an underfitted model would have that bias because it's averaged it out so if you have um five people from India and 10 people from um Africa and 20 people from the US you've created a bias uh because it's looking at the 20 people and you only have a small amount of data to work with size of training data used is not enough uh that goes with the size I was just talking about so we have a model with a high bias we have size of training data used it's not enough the model is too simple again this a one straight line 2:25:35 through all the data when it needs has a slight shift to it for other reasons so what is a good fit uh a linear curve that best fits the data is neither overfitting or underfitting models but is just right and of course we have the nice examples here where we have overfitting lines going up and down every Point's starting to be include glued underfitting uh the line really is off from where the data is and then a good fit is got to get rid of that minimize that um error coming through regularization is taking the guess work out you're looking at this graph and you're going oh which one is that really overfit or is that underfit that's pretty hard to tell so we talk about about regularization regularization techniques are used to calibrate the linear regression models and to minimize the adjusted loss function and prevent overfitting or underfitting so what that means uh in 2:26:36 this case we're going to go ahead and take a look at a couple different things we're going to look at regularization which we'll start with a linear model we'll look at the Ridge regularization and the lasso regularization and these models are just like uh just like we did the the um MLP the multi-layered positron and the sklearn module you could bring in the ridge module and you can bring in the lassle module so when we talk about um Ridge regression it modifies the overfitted or underfitted models by adding the penalty equivalent to the sum of the squares of the magnitude of the coefficients and so we have a cost function equals loss equals Lambda time the sum of W squared or the absolute value of w depending on how you're doing it now remember we talked about error where we either Square it or we absolute value it uh because that removes a plus or minus sign on there and there's reasons to do it either way but it is more common to square the 2:27:36 value and then we have our uh um in this case the Lambda is going to be the penalty for the errors we've thrown in a Greek character for you just to confuse everybody and W is the slope of the curve of the line so uh we're going to look at this and we're going to draw a line this is going to be like a linear regression model so if you had in sklearn you could import just a standard linear regression model uh it would plot this line across whatever data we're working on and we look at this and of course we're just extrapolating this um I know they use some specific data but don't want to get into the actual domain and so for a linear regression line let's consider two points that are on the line and we'll go ahead and have a lot equals 0 uh considering the two points on the line we'll go and do Lambda equal 1 we'll set our W is going to be 1.4 then the cost function equals 0 + 1 * 1.42 which equal 2:28:37 1.96 uh so really don't get caught up too much in the math on this other than understanding um that this is something that's very easy for a computer to calculate and if you ever see the loss plus the plus the Lambda time W the sum of w^2 and then let's say we have a ridge regression line and it does this we go ahead and plot it and we do the calculations on the data and for the ridge regression let's assume a loss equals 3^2 + 2^2 = .13 so when they put all the calculations through of the two points we end up with the 0.0 62 uh so we now had a linear regression model we now had a ridge regression model the rid regression model it plots a little differently than the standard linear regression model and comparing the two models with all the data points we can see that the ridge regression line fits the model more accurately than the linear regression line and I find this true on a lot of 2:29:38 data I work with I'll end up using either the ridge regression model or the lasso uh Mars regression model for fitting um especially dealing with a lot of like uh stock markets daily um setup they come out slightly better uh you get a slightly better um fit and so we have our lasso we just talked about lasso coming in here and the cost function equals uh um instead of doing a squared we're just going to do the absolute value and so if you remember this is where Ridge regression changes where's my Ridge regression model uh we're squaring the value here and if you look at this we're not squaring the value we're just finding the ABS absolute value on here and so the loss of the of the squared individuals um and here is our Lambda symbol again penalty for errors and W equals the slope of the curve and comparing the two models with all the data points we can see that the 2:30:39 lasso regression line fits the model more accurately than the linear regression line and this is like I said I I use these two models a lot uh the ridge and this is important this is this is kind of the meat of the matter how do you know which one to use some of it is you just do it a bunch of times and then you figure it out uh Ridge regularization is useful when we have many variables with relatively smaller data samples the bottle does not encourage convergence toward zero but is likely to make them closer to zero and prevent overfitting the Lasser regularization model is preferred when we are fitting a linear model with with fewer variables so in the LA in the iris thing we had four or five variables as they measured the different Leaf pieces uh you might be doing the measurements on the cancer project which has 36 different variables um so as we get down to the iris with four variables lassar will 2:31:40 probably uh work pretty good where you might use the ridge regularization with more model with if you have something significantly larger and it encourages the coefficients of the very Ables to go towards zero because of the shape of the constraint which is an absolute value and with any of this we want to go ahead and uh do a demo and lasto and ridge regression so let's take a look and see what that looks like in our code and bring up our Jupiter notebook we'll start with our Imports pandas as PD import numpy as NP import matplot Library as PLT um SK learn we're going to import our data sets this kind of more generic um we use just import one data set instead of all of them but you know quick and dirty when you're putting some of these together uh we have our sklearn model selection we're going to import our train test split for splitting our data up and then we'll bring in our linear regression model and we'll go ahead and run these just to load them up and then load our data set we were just talking about that 2:32:42 uh you could just um um have imported the load Boston and Boston data set in there instead of loading all the data sets um and then once we've loaded our data set we want to go ahead and take a look at that data and see what we got here let me just go and pop that down there and go a and run it and so we've G ahead and taken our uh Boston uh data we're going to look we put it into our pandas data frame um the Boston data set and then the Boston columns so we want to see what's going on with them um we have our Target we have the house price uh Etc and so our x equals Boston um iocation now remember in pandas the new updates to pandas they want iocation if you're going to pull data we used to be able to leave this off but it does something different it creates a slice versus a direct um setup so make sure using that iocation and the I the output 2:33:44 so this is all just bringing our data together and we can see here if we do uh we print the Boston um Panda's head we can see here all of our different um aspects we're looking for and if you're following the X and the Y the x is um everything except for the last column where Y is uh all the it's that's what this means all the rows except for the last column and then Y is all the rows but just the last column so Y is our house price and the x is the Crim xan industry Chaz Knox and all these other different um statistics they've collected for house sales in Boston there we go oops control ah so we'll go ahead and split our data uh XT train and our x uh test YT train y test equals the train test split which 2:34:44 we imported and we have our Boston um you could have easily used the and Y on here as opposed to Boston um iocation and we'll create our test size we're going to take 25% of the data and put it in as a test uh and then we'll go ahead and run this uh need an extra drink there uh so we have our train and test and then of course the print the train data shape I love doing this kind of thing whenever I'm working with this data print out the shape make sure everything looks correct uh so that we have 127 by 13 and 127 by 1 379 by 13 they should match and if there if the the data sets are not quite matching then you know something's wrong and you're going to get all those errors I don't know how many times I've gone through here and it's dropped a row on one of them and not on the other or something weird is happen when I'm cleaning the data this 2:35:45 is pretty straightforward and simple cuz the data comes in a nice prepackage it's all cleaned for you so let's go ahead and apply apply the multiple linear regression model and uh we'll call this l r lreg linear regression and we're going to go ahead and fit that linear regression model to X train and Y train then we'll generate the prediction on the test Set uh so here's our lre y predict with our X test going into the prediction and let's calculate that mean square error MSE I told you you'll see MSE used a lot um people use it in variables and things like that it's pretty common and we get our mean squared error equals uh this is just the basic formula we've already been talking about what's the difference squared uh then we look for the average of that we'll go ahead and just run this and you can see when we get through the end of this we have our mean square 2:36:47 error on test we have have our total and then we have each column coming down and at this point unless you really know the data you're working with it's not going to mean a whole lot so if it's in your domain you might be know what you're looking at when you see see these kinds of numbers coming up uh but if it's not it's just a bunch of numbers and that's okay at least that's okay for this demo uh and then we're going to go ahead and plot these so we can see what's going on and this is always kind of fun it's always nice to have a nice visual of what you're looking at and you can see here um when we plot the coefficient scores on here and we the guys in the back did a great job putting some pretty colors together making it look nice and setting up the columns you can see here uh your nox has like just a huge coefficient um when I look at a table like this I look for what has very little um different coefficients they're not using a huge 2:37:48 change and what has huge changes and that Flags you for all kinds of things as you're working with a data but it depends so much on the domain you're working with these are great things though as H just a quick look to see what's going on with your data and what you're looking for and of course once we look at this now our motive is to reduce the coefficient score so now we want to take these and and uh bring them down as much as we can and for that we're going to work with the ridge regression on here so let's start by go ahead and we're going to import our Ridge model the ridge regression from the sklearn library or the S kit and we're going to go ahead and train the model so here's our Ridge R equals Alpha equal 1 and I mentioned that earlier um when I work with the ridge model you'll see Alpha equals 1 if you set Alpha equal to zero that's a standard linear regression model so you have Alpha equal 1 2 3 4 and you usually use 1 2 or 3 4 and a standard integer on there 2:38:50 and we'll go ahead and fit the ridge model on there with our X train and our YT train data generate a prediction for that um for our X test and we'll calculate the mean square error uh just like we did before this should all look familiar and we'll go ahead and print that out and we'll look at the uh Ridge coefficients for our data and see what that looks like now if I jump up and down between these two you'll get a headache uh you'll still see the nox value let's just look at the nox because that was the biggest value was a minus 9 here and if we go back up here the nox value is a minus8 so right off the bat I'm seeing a huge change um in the biggest coefficient there uh so if we're going to do that nice setup we want to go ahead and just print it and see what that looks like 2:39:53 here we go and we've uh set up our um plot subplots and again the team put together some nice color so it makes it look good we're doing an xar based on the columns and our um L regress coefficients color equals color a spine bottom and so forth uh so just puts together a nice little graph and you're starting to see uh one this when you compare this if you put on the same graph is this one up here this is up here at -8 this is at Min - 9 and so this graph is half the size of the graph above uh the same thing with these values here they might look the same but they're actually all uh almost half the value on here and then finally you can do the same thing for the lasso regression this will all look uh very similar as far as what we worked on before and I'm just going to ahead and print that on here and run it and again let's 2:40:54 go up to uh KNX look where KNX is it's all the way down to zero and if we look at our next biggest coefficient is minus8 and really here's our 22.73 we go up here um 16.7 and we go up here and we look at the same number uh 16 69 and so we look at this uh if I was running this and doing a working on a project with this I would look at these numbers I start with the 2:41:48 16.69% we might know that uh the knock value is Central and the other ones aren't quite as good and so we might start looking at just certain setups like what is our what is this particular coefficient because it might have a certain meaning to us and so forth and so you got to look at all those different thing um items in there again but the bottom dollar is our first model did better than our other two models our mean square error on the test set um continues to come down on this we're going to cover reinforcement learning today and what's in it for you we'll start with why reinforcement learning we'll look at what is reinforcement learning we'll see what the different kinds of learning strategies are that are being used today in computer models under supervised versus unsupervised versus reinforcement we'll cover important terms specific to reinforcement learning we'll talk about markov's decision process and we'll take a look at a reinforcement learning example well we'll teach a tic-tac-toe how to play 2:42:50 why reinforcement learning training a machine learning model requires a lot of data which might not always be available to us further the data provided might not be reliable learning from a small subset of actions will not help expand the vast realm of solutions that may work for a particular problem and you can see here we have the robot learning to walk um very complicated setup when you're learning how to walk and you'll start asking questions like if I'm taking one step forward and left what happens if I pick up a 50 PB object how does that change how a robot would walk these things are very difficult to program because there's no actual information on it until it's actually tried out learning from a small subset of actions will not help expand the vast realm of solutions that may work for a particular problem and we'll see here it learned how to walk this is going to slow the growth that technology is capable of machines need to learn to perform actions by themselves and not just learn 2:43:50 off humans and you see the objective climb a mountain real interesting point here is that as human beings we can go into a very unknown environment and we can adjust for it and kind of explore and play with it most of the models the non-reinforcement models in computer uh machine learning aren't able to do that very well uh there's a couple of them that can be used or integrated see how it goes is what we're talking about with reinforcement learning so what is reinforcement learning reinforcement learning is a subbranch of machine learning that trains a model to return an Optimum solution for a problem by taking a sequence of decisions by itself consider a robot learning to go from one place to another the robot is given a scenario and must arrive at a solution by itself the robot can take different paths to reach the destination it will know the best path by the time taken on each path it might 2:44:52 even come up with a unique solution all by itself and that's really important as we're looking for Unique Solutions uh we want the best solution but you can't find it unless you try it so we're looking at uh our different systems or different model we have supervised versus unsupervised versus reinforcement learning and with the supervised learning that is probably the most controlled environment uh we have a lot of different supervised learning models whether there's linear regression neural networks um there's all kinds of things in between decision trees the data provided is labeled data with output values specified and this is important because when we talk about supervised learning you already know the answer for all this information you already know the picture has a motorcycle in it so you're supervised learning you already know that um the outcome for tomorrow for you know going back a week you're looking at stock you can already have like the graph of what the next day looks like so you have an answer for it and you have labeled data which is used 2:45:52 you have an external supervision and solves Problems by mapping labeled input to know an output so very controlled unsupervised learning and unsupervised learning is really interesting because it's now taking part in many other models they start with an you can actually insert an unsupervised learning model um in almost either supervised or reinforcement learning as part of the system which is really cool uh data provided is unlabeled data the outputs are not specified machine makes its own predictions used to solve association with clustering problems unlabeled data is used no supervision solves Problems by understanding patterns and discovering output uh so you can look at this and you can think um some of these things go with each other they belong together so it's looking for what connects in different ways and there's a lot of different algorithms that look at this um when you start getting into those there's some really cool images that come up of what unsupervised learning is 2:46:55 how it can pick out say uh the area of a doughnut one model will see the area of the doughnut and the other one will divide it into three sections based on this location versus what's next to it so there's a lot of stuff that goes in with unsupervised learning and then we're looking at reinforcement learning probably the biggest industry in today's market uh in machine learning or growing Market it's very it's very infant stage uh as far as how it works and what it's going to be capable of the machine learns from its environment using rewards and errors used to solve reward based problems no predefined data is used no supervision follows Trail and error problem solving approach uh so again we have a random first you start with a random I try this it works and this is my reward doesn't work very well maybe or maybe doesn't even get you where you're trying to get it to do and you get your reward back and then it looks at that and says well let's try something else and it starts to play with these different things finding the best route so let's take a look at 2:47:56 important terms in today's reinforcement model and this has become pretty standardized over the last uh few years so these are really good to know we have the agent uh agent is the model that is being trained via reinforcement learning so this is your actual uh entity that has however you're doing it whether you're using a neural network or a a q table or whatever combination thereof this is the actual agent that you're using this is the model and you have your environment uh the training situation that the model must optimize to is called its environment uh and you can see here I guess we have a robot who's trying to get a chest full of gems or whatever and that's the output and then you have your action this is all possible steps that can be taken by the model and it picks one action and you can see here it's picked three different uh routes to get to the chest of diamonds and gyms we have a state the current position condition returned by the model and you could look at this uh if 2:48:58 you're playing like a video game this is the screen you're looking at uh so when you go back here uh the environment is a whole game board so if you're playing one of those Mobius games you might have the whole game board going on uh but then you have your current position where are you on that game board what's around that what's around you um if if you were talking about a robot the environment might be moving around the yard where it is in the yard and what it can see what input it has in that location that would be the current position condition returned by the model and then the reward uh to help the model move in the right direction it is rewarded points are given to it to appraise some kind of action so y you did good or if uh didn't do as good trying to maximize the reward and have the best reward possible and then policy policy determines how an agent will behave at any time it acts as a mapping between action and present State this is part of the model what what is your action that 2:49:59 you're you're going to take what's the policy you're using to have an output from your agent one of the reasons they separate uh policy as its own entity is that you usually have a prediction um of a different options and then the policy well how am I going to pick the best based on those predictions I'm going to guess at different options and we'll actually weigh those options in and find the best option we think will work uh so it's a little tricky but the policy thing is actually pretty cool how it works let's go ahe and take a look at a reinforcement learning example and just in looking at this we're going to take a look uh consider what a dog um that we want to train uh so the dog would be like the agent so you have your your puppy or whatever uh and then your environment is is going to be the whole house or whatever it is where you're training them and then you have an action we want to teach the dog to fetch so action equals fetching uh and then we have a little 2:51:00 biscuit so we can get the dog to perform various actions by offering incentives such as a dog biscuit as a reward the dog will follow a policy to maximize this reward and hence will follow every command and might even learn new actions like begging by itself uh so you have B you know so we start off with fetching it goes oh I get a biscuit for that it tries something else and you get a handshake or begging or something like that and it goes oh this is also reward-based and so it kind of explores things to find out what will bring it as biscuit and that's very much like how a reinforced model goes is it uh looks for different rewards how do I find can I try different things and find a reward that works the dog also will want to run around and play explorers environment uh this quality of model is called exploration so a little Randomness going on in Exploration and explores new parts of the house climbing on the sofa doesn't get a reward in fact it usually gets kicked off the 2:52:00 sofa so let's talk a little bit about markov's decision process uh markov's decision process is a reinforcement learning policy used to map a current state to an action where the agent continuously interacts with the environment to produce new Solutions and receive Rewards and you'll see here's all of our different uh uh vocabulary we just went over we have our reward our state our agent our environment and our action and so even though the environment kind of contains everything um that you you really when you're actually writing the program your environment's going to put out a reward in state that goes into the agent uh the agent then looks at this uh state or it looks at the reward usually um first and it says okay I got rewarded for whatever I just did or I didn't get rewarded and then looks at the state then it comes back and if you remember from policy the policy comes in um and then we have a reward the policy is that part that's 2:53:01 connected at the bottom and so it looks at that policy and it says hey what's a good action that will probably be similar to what I did or um sometimes they're completely random but what's a good action that's going to bring me a different reward so taking the time to just understand these different pieces as they go is pretty important in most of the models today um and so a lot of them actually have templates based on this you can pull in and start using um pretty straightforward as far as once you start seeing how it works uh you can see your environment send says hey this is the agent did this if you're a character in a game this happened and it shoots out a reward in a state the agent looks at the reward looks at the new state and then takes a little guess and says I'm going to try this action and then that action goes back into the environment it affects the environment the environment then changes depending on what the action was and then it has a 2:54:02 new state and a new reward that goes back to the agent so in the diagram shown we need to find the shortest path between note A and D each path has a reward associate with it and the path with a maximum reward is what we want to choose the nodes AB CD denote the nodes to travel from node uh A to B is an action reward is the cost of each path and policy is each path taken and you can see here a can go uh to b or a can go to C right off the bat or you can go right to D and if you explored all three of these uh you would find that a going to D was a zero reward um a going to C and D would generate a different reward or you could go AC b d there's a lot lot of options here um and so when we start looking at this diagram you start to realize that even though uh today's reinforced learning models do really good at um finding an answer they end up trying almost all the different 2:55:02 directions you see and so they take up a lot of work uh or a lot of processing time for reinforcement learning they're right now in their infant stage and they're really good at solving simple problems and we'll take a look at one of those in just a minute in a tic tac toe game uh but you can see here uh once it's gone through these and it's explored it's going to find the ACD is the best reward it gets a full 30 points for it so let's go ahead and take a look at a reinforcement learning demo uh in this demo we're going to use reinforcement learning to make a tic tac toe game you will be playing this game Against the Machine learning model and we'll go ahead we're doing it in Python so let's go ahead and go through my always uh not always actually have a lot of python tools let's go through um Anaconda which will open up a Jupiter notebook seems like a lot of steps but it's worth it to keep all my stuff separate and it's also has a nice display when you're in the Jupiter notebook for doing 2:56:02 python so here's our Anaconda Navigator I open up the notebook which is going to take me to a web page and I've gone in here and created a new uh python folder in this case I've already done it and enabled it to change the name to tic-tac-toe uh and then for this example uh we're going to go ahead and import a couple things we're going to um import nump as NP we'll go ahead and import pickle nump of course is our number array and then uh pickle is just a nice way sometimes for storing uh different information uh different states that we're going to go through on here uh and so we're going to create a class called State we're going to start with that and there's a lot of lines of code to this uh class that we're going to put in here don't let that scare you too much there's not as much here um it looks like there's going to be a lot here but there really is just a lot of setup going on in the in our class date and so we have up here we're going to initialize it um we have our board um 2:57:06 it's a tic teac toe board so we're only dealing with nine spots on the board uh we have player one player two uh is in we're going to create a board hash uh we'll look at that in just a minute we're just going to store some information in there symbol of player equals one um so there's a few things going on as far as the initialization uh then something simple we're just going to get the hash um of the board we're going to get the information from the board on there which is uh columns and rows we want to know when a winner occurs uh so if you get three in a row that's what this whole section here is for um me go ahead and scroll up a little bit and you can get a copy of this code if you send a note over to Simply learn we'll send you over um this particular file and you can play with it yourself and see how it's put together I don't want to spend a huge amount of time on this uh because this is just some real General python coding uh but you can see here we're just going through all the rows and you 2:58:09 add them together and if it equals three three in a row same thing with columns uh diagonal so you got to check the diagonal that's what all this stuff does does here is it just goes through the different areas actually let me go ahead and put there we go um and then it comes down here and we do our sum and it says true uh minus three just says did somebody win or is it a tie so you got to add up all the numbers on there anyway just in case they're all filled up and next we also need to know available positions um these are ones that don't no one's ever used before this way when you try something or the computer tries something uh it's not going to give it an illegal move that's what the available positions is doing uh then we want to update our state and so you have your position going in we're just sending in the position that you just chose and you'll see there's a little user interface we put in there you P pick the row and column in there and again I mean this is a lot of 2:59:10 code uh so really it's kind of a thing you'd want to go through and play with a little bit and just read through it get a copy of it great way to understand how this works and here is a given reward um so we're going to give a reward result equals self winner this is one of the hearts of what's going on here uh is we have a result self. winner so if there's a winner then we have a result if the result equals one here's our feedback uh if it doesn't equal one then it gets a zero so it only gets a reward in this particular case if it wins ends and that's important to know because different uh systems of reinforced learning do rewarding a lot differently depending on what you're trying to do this is a very simple example with a a 3X3 board imagine if you're playing a video game uh certainly you only have so many actions but your environment is huge you have a lot going 3:00:11 on in the environment and suddenly a reward system like this is going to be just um is going to have to change a little bit it's going to have to have different rewards and different setup and there's all kinds of advanced ways to do that as far as weighing you add weights to it and so they can add the weights up depending on where the reward comes in so it might be that you actually get a reward in this case you get the reward at the end of the game and I'm spending just a little bit of time on this because this is an important thing to note but there's different ways to add up those rewards it might have like if you take a certain path um the first reward reward is going to be weighed a little bit less than the last reward because the last reward is actually winning the game or scoring or whatever it is so this reward system gets really complicated on some of the more advanced uh setups um in this case though you can see right here that they give a um a 0.1 and a 0 five reward um just for getting a picking the 3:01:11 right value and something that's actually valid instead of picking an invalid value so rewards again that's like key it's huge how do you feed the rewards back in uh then we have a board reset that's pretty straightforward it just goes back and resets the board to the beginning because it's going to try out all these different things while it's learning it's going to do it by trial and error so you have to keep resetting it and then of course there's the play we want to go ahead and play uh rounds equals 100 depends on what you want to do on here um you can set this different you obviously set that to higher level but this is just going to go through and you'll see in here uh that we have player one and player two this is this is the computer playing itself uh one of the more powerful ways to learn to play a game or even learn something that isn't a game is to have two of these models that are basically trying to beat each other and so they they keep finding explore new things this one works for 3:02:12 this one so this one tries new things it beats this we've seen this in um chess I think it was a big one where they had the two players in chess with reinforcement learning uh was one of the ways they train one of the top um computer chess playing algorithms uh so this is just what this is it's going to choose an action it's going to try something and the more it triy stuff um the more we're going to record the hash we actually have a board hash where they self get the hash setup on here where stores all the information and then once you get to a win one of them wins it gets the reward uh then we go back and reset and try again and then kind of the fun part we actually get down here is uh we're going to play with a human so we'll get a chance to come in here and see what that looks like when you put your own information in and then it just comes in here does the same thing it did above it gives it a reward for its things um or sees if it wins or ties um looks at available positions all that kind of fun 3:03:13 stuff and then finally we want to show the board uh so it's going to print the board out each time really um as an integration is not that exciting what's exciting uh in here is one looking at this reward system whoops Play One More up the reward system is really the heart of this how do you reward the different uh setup and the other one is when it's playing it's got to take an action and so what it chooses for an action is also the heart of reinforcement learning how do we choose that action and those are really key to right now where reinforcement learning is um in today's uh technology is uh figuring this out how do we reward it and how do we guess the next best action so we have our uh environment and you can see the environment is we're going to be or the state uh which is kind of like what's going on we're going to return the state depending on what 3:04:13 happens and we want to go ahead and create our agent uh this clay are player so each one is let go and grab that and so we look at a class player um this is where a lot of the magic is really going on is what how is this player figuring out how to maneuver around the board and then the board of course returns a state uh that it can look at and a reward uh so we want to take a look at this we have a name uh self State this is class player and when you say class player we're not talking about a human player we're talking about um just a a the computer players and this is kind of interesting so remember I told you depending on what you're doing there's going to be a Decay gamma um explore rate uh these are what I'm talking about is how do we train it um as you try different moves it gets to the end the first move is important but it's not as important as the last one and so you could say that the last one 3:05:14 has the heaviest weight and then as you as you get there the first one the the first move gives you a five reward the second gives you a two reward and the third one gives you a 10 reward because that's the final ending you got it the 10's going to count more than the first step uh and here's our uh we're going to you know get the board information coming in and then choose an action this was the second part that I was talking about that was so important uh so once you have your training going on we have to do a little Randomness and you can see right here is our NP random um uniform so it's picking out a random number take a random action this is going to just pick which row and which colum it is um and so choosing the action this one you can see we're just doing random States um Choice length of positions action position and then it skips in there and takes a look at the board uh for p and positions you it's actually storing the different boards each time you go through so it it has a 3:06:14 record of what it did so it can properly weigh the values and this simply just ains a hash State what's the last state pin it to the uh to our states on here here's our feedback reward so the reward comes in and it's going to take a look at this and say is it none uh what is the reward and here is that formula remember I was telling you about up here um that was important because it has Decay gamma times the reward this is where as it goes through each step and this is really important this is this is kind of the heart of this of what I was talking about earlier uh you have step one and this might have a a reward of two you have step two I should probably should have done ABC this has a step three uh step four so on till you get to step in and this might have a reward of 10 uh so reward of 3:07:16 10 we're going to add that but we're not adding uh let's say this one right here uh let's say this reward here right before 10 was um let's say it's also 10 that just makes the the math easy so we had 10 and 10 we had 10 this is 10 and 10 n whatever it is but it's time it's 0.9 uh so instead of putting a full 10 here we only do nine that's uh 0.9 times 10 and so this formula um as far as the Decay times the reward minus the cell State value uh it basically adds in it says here's one or here's two I'm sorry I should have done this ABC would have been easier uh so the first move goes in here and it puts two in here uh then we have our s uh setup on here you can see how this gets pretty complicated in the math but this is 3:08:16 really the key is how do we train our states and we want the the final State the win to get the most points if you win you get most points um and the first step gets the least amount of points so you're really training this almost in Reverse you're training you're training it from the last place where you have like it says okay this is now where I need to sum up my rewards and I want to sum them up going in reverse and I want to find the answer in Reverse kind of an interesting uh uh play on the mind when you're trying to figure F this stuff out and of course we want to go ahead and reset the board down here uh and save the policy load policy these are the different things that are going in between the agent and the state to figure out what's going on let's go ahead and load that up and then finally we want to go ahead and create a human player and the human player is going to be a little different uh in that uh you choose an action row and column here's 3:09:17 your action uh if action is if action in positions meaning positions that are available uh you return the action if not it just keeps asking you until you get an action that actually works and then we're going to go ahead and append to the hash state which uh we don't need to worry about because it Returns the action up here and feed forward uh again this is because it's a human um at the end of the game bat propagating update State values this part isn't being done because it's not programming uh the model uh the model is getting its own rewards so we've gone ahead and loaded this in here uh so here's all our pieces and the first thing we want to do is set up uh P1 player one uh P2 player two and then we're going to send our players to our state so now it has P1 P2 and it's going to play and it's going to play 50,000 rounds now we can 3:10:20 probably do a lot less than this and it's not going to get the full results in fact you know what uh let's go ahead and just do five um just to play with it because I want to show you something here oops somewhere in there I forgot to load something there we go I must have forgot to run this run oops forgot a reference there for the board rows and columns 3x3 um there is actually in the state it references that we just tack it on on the end it was supposed to be at the beginning uh so now I've only set this up with um see where are we going here I've only set this up to train five times and the reason I did that is we're going to uh come in and actually play it and then I'm going to change that and we can see how it differs on there there we go did you make it through a run and we're going to go ahead and save the 3:11:21 policy um so now we have our player one and our player two policy uh the way we set it up it has two separate policies loaded up in there and then we're going to come in here and we're going to do uh player one is going to be the computer experience rate zero load policy one human player human and we're going to go ahead and play this I remember I only went through it um uh just one round of training in fact minimal training and so it puts an X there and I'm going to go ahead and do row 0o column one you can see this is very uh basic on here and so I put in my zero and then I'm going to go zero block it zero zero and you can see right here it let me win uh just like that I was able to win zero two and woo human wins so only trained it five times we're going to run this again and this time uh instead of 3:12:23 five let's do 5,000 or 50,000 I think that's what the guys in the back had and this takes a while to train it this is where reinforcement learning really falls apart look how simple this game is we're talking about uh 3x3 set of columns and so for me to train it on this uh I could do a q table which would take which would go much quicker um you could build a quick Q table with almost all the different options on there and uh you would probably get a the same result much quicker we're just using this as an example so when we look at reinforcement learning you need to be very careful what you apply it to it sounds like a good deal until you do like a large neural network where you're doing um you set the neural network to a learning increment of one so every time it goes through it learns and then you do your action so you pick from the learning uh setup and 3:13:24 you actually try actions on the learning setup until you get the what you think is going to be the best action so you actually feed what you think is right back through the neural network there's a whole layer there which is really fun to play with and then it has an output well think of all those processes I mean that is just a huge amount of work it's going to do uh let's go ahead and Skip ahead here give it a moment it's going to take a a minute or two to go ahead and run now to train it uh we will went ahead and let it run and it took a while this this took um I got a pretty powerful processor and it took about five minutes plus to run it and we'll go ahead and uh run our player setup on here oops I brought in the last whoops I brought in the last round so give me just a moment to redo the policy save there we go I forgot to save the policy back in there and then go ahead and run our player again so we we've saved the 3:14:27 policy and then we want to go ahead and load the policy for P1 as the computer and we can see the computer's gone in the bottom right corner I'm going to go ahead and go uh one one which is the center and it's gone right up the top and if you have ever played tic tactoe you know the computer has me uh but we'll go ahead and play it out out row zero column two there it is and then it's gone here and so I'm going to go ahead and go row 01 two no 01 there we go and column zero that's where I wanted oh and it says I okay you your action there we go boom uh so you can see here we've got a didn't catch the win on this it said Tai um kind of funny they didn't catch the win on there but if we play this a bunch of times you'll find it's going to win more and more the more we train it the more the reinforcement happens this lengthy training process uh is really the stopper on reinforcement 3:15:28 learning as this changes reinforcement learning will be one of the more powerful uh packages evolving over the next decade or two in fact I would even go as far as to say it is the most important uh machine learning tool and artificial intelligence tool out there as it learns not only simple Tic Tac Toe board but we start learning environments and the environment would be like in language if you're translating a language or something from one language to the other so much of it is lost if you don't know the context it's in what's the environments it's in and so being able to attach environment and context and all those things together is going to require reinforcement learning to do today we are going to explore K neighbors or KNN which is one of the most popular algorithms in data science python is a powerful tool for data science and KNN is great for classifying data by predicting the category of sample base on its closest neighbor this algorithms is used in many field like healthcare finance and agriculture 3:16:31 helping us make decision based on data the best part it is really easy to use you just need to pick a number for K and choose a distance function to compare data points however K&N has it downsides it doesn't work well with the last data set and it require proper scaling of the data to get accurate result in this video we will show you how Cann work with real data set the ISS data set we'll walk you through simple python code and demonstrate how to find the best ke value to maximize your models accuracy so stay tuned to CNN in action so welcome to the demo part so I'm here using Google collab so you can use any of your favorite ID like jupyter notebook intellig visual code Studio anything okay so let me rename this file as km classification Okay cool so let me tell 3:17:31 you that KNN can be used for the classification regression predictive problems so can and falls in the supervised learning family of algorithms okay so we will measure the distance between the K neighbors and the first step will be we will choose the number of K of neighbors then uh we'll take the K nearest neighbors of the new data point according to your distance metric and the step three will be our among these case neighbor count the number of data points of each category okay step four we will assign the new data points to the category where you counted the mosters okay so let's start first let's import some Library import numpy as NP and let's import FAS SPD so everyone knows what is numai and the pandas okay so naai is the library for the Python programming language adding support for you know large multi-dimensional arrays okay 3:18:33 along with the large collection of uh what to say uh high level mathematical functions and whereas pandas is a software Library written for the Python programming language for the data manipulation and Analysis uh all the data frames and uh data structure it offers for the manipulating numerical tables okay and the time series you can so moving forward uh we'll import our data set so you can download the data set from the description box below okay data sets to pd. read uncore CSV the data name is iris. CSV okay this is how you read uh your data in Python okay yeah data set is loaded so data set. shape shape okay yeah so data set do shape is used for how many numbers of 3:19:34 rows and columns present in your data set okay 150 rows and six columns so let me tell you brief about data set this Isis data set so is data set include three Idis species with uh 50 samples each as well as uh some properties about each FL so one FL species is linearly separable from the other two you can say but the other two are not linearly separable from each other okay and this shape I told you we can get a quick idea of how many instances of rows and columns are present in our data set so let's see our data set data set do head so head is used for uh you know by you can see top five rows of your data set using head and if you will use tail you instead of head you can see the last five rows of your data set cool okay see columns are ID SLE length SLE width petal length petal width and this 3:20:35 pieces cool then moving forward let's describe our data sets so these are the basics uh basic function okay of python you you can say so data set do described what describes do is it will give you count of all the rows mean value standard deviation value minimum value what is the 25% okay of all the values in the particular row what is the 50% what is the 75% what is the maximum maximum is 150 you can say okay and 25% of 150 is 38.25 this okay of all the columns if it is normal uh you know character so it won't give you any data okay cool yeah so moving forward uh let's now take a look at the number of instances to belong to each classes okay so we will write data chat 3:21:35 dot Group by species do size okay uh this s is capital that's why it's showing the error yeah so you can see Iris satsa are 50 IR versical are 50 and virin is 50 okay and the data type is integer cool so as we can see data set contains six columns like ID saple length sapple width and petal length petal width and spe the actual features are described by columns 1 to 4 the last columns labels of samples okay so firstly we need to split data into two arrays like X features and Y labels so how we will do this by writing code like feature underscore columns equals to sample lens sample width P length P width just remember you are writing correct name okay then xals to data 3:22:40 set feature underscore columns okay dot values then y equals to data set species do values cool then let me run it so this is uh how we can split the data set okay into two arrays X and the Y okay in X there are feature columns these four columns are there and in y species column is there okay and what is the species column this SATA vinica and all this cool then now we will do label en coding so as you can see labels are cical K inverse classifier does not accept string labels so we need to use label encoder to transform them into numbers okay then Iris SATA correspond to zero Iris ver correspond to one and Iris vinica 3:23:42 correspond to two Okay 0 1 2 cool so how can that from sklearn do preprocessing import label encoder okay so what we'll do label and coder transform them into numbers okay so I will write Al equals to label encoder okay my bad able encoder okay then yal to elate transform y okay so now I will run it yeah correct so splitting data set into training set and the test set now okay so now we'll split data set into training set and test set to check later on whether or not a classifier work correctly or not okay so here I will write from skar dot not cross I will trite here 3:24:43 skarn do model selection import import train TR split so what is train TR split train test split is a model validation procedure that reveals how your model performs on your new data okay and what is xtrain XS y train y s in Python okay let me first write it and then I will let you know okay then I will write here xra comma X test comma y train comma y test okay equals to train test split then X comma y comma test size 0.2 and the random is this okay okay some error came okay underscore model 3:25:44 section yeah so what is X train X test y train y test okay so X train and Y train sets are used for training and fitting the model okay so the X test and the Y test are the set used for testing the model and it's predicting the right output level okay so here you can see test size is 0.2 means 80% is for testing oh sorry 80% is for training and 20% is for testing for the new data cool yeah so now we will see some let's do some data visualization okay so here I will write import met plot lib do pyplot as PLT then import cbon as SNS then here I will write person M plot 3:26:45 live in line so what is matplot lib matplot lib is a plotting library for the Python programming language and its numerical mathematic extension okay so numpy it provides an object API for embedding plots into application using generating purpose GUI tool Gates like kter and python qt or gtk okay whereas cbon cbor is a library for making statical graph in Python it builds on the top of M plot lip and creates closely with Panda's data structure okay cbon helps us to explore and understood the data okay so I will run it here I will write from pandas do plotting import parallel coordinates then PLT dot figure figure 3:27:49 size should be 15 comma 10 okay then parallel coordinates okay then data set dot drop I don't need ID comma X is one okay and comma species then PLT do title and let parallel coordinates plot okay and you can give some font size equal to 20 then font vit equals to okay let's say bold only bold then PLT do X 3:28:50 label then features comma font size to 15 then PLT do y LEL then I'll write here features values comma font size = to 15 okay then PLT do Legend then loc to one comma I will WR here frame on equals to True comma Shadow = to True comma face color equals to White T should be 3:29:51 capital and comma Edge color equals to okay PLT do show okay some error is there PLT dot figure size okay spelling mistake okay one more pl. Legend okay face color okay some spelling sake so yeah let me make it you output in full screen yeah so parallel coordinates is a plotting technique for plotting you know multivar data so it allows one to see clusters in the data and to estimate other stat visually so using parel coordinate points uh you know are represented as the connected line segments as you can see okay and each vertical line represent one attribute and one set of connected line 3:30:52 segments represent one data point okay and points that tend to Cluster will appear closer together okay so this this color is Iris satsa and this is Iris ver and this purle one is Iris vinica as you can see in the legend Okay cool so moving forward let's create another graph Andre curves okay so here I will write from pandas do plotting import sker no plotting import Andre curves okay then PLT do figure it's AI based so it's giving me suggestions suggestion suggestions okay so sometimes suggestions are good but not always yeah let's carry on figure size is 15 comma 10 then 3:31:56 Andrew cures then I will add data set do drop idx spes and PLT and reest plot okay fine okay let me add we don't need X label and all let me add Legend PLT do Legend then same Lo equals to 1 then proposition size then I will write here size is 15 frame on = to 2 comma Shadow equals to true so this is uh totally based upon you if if you want to add legend or not or you can skip if you want to skip you can skip okay face color equals to White 3:32:57 then H Color equals to Black okay then PLT do show yeah so no error is there so let me first view output in full screen okay so Andrew curves these are the end curves okay you can see the graph in the curve so end curve allow one to plot multivar data as a large number of Curves so that are created using the attributes of samples okay as coefficient for for year series okay so by coloring these curves differently for each class it is possible to visualize data clustering so curves belongs belonging to samples of the same class will usually be closer together and they form large structure okay as you can see here and this is the legend why we are setting the four color it should be white and yeah and the edge color should be black okay and frame on 3:33:59 Shadow should be there you can see the shadow okay like this if you want to skip you can skip this part legend legend part okay but like it's good to have it's good practice to have this cool so let's create one a small pair plot okay so I will write here PLT dot figure then I will write SNS do pairplot then I will write data set dot drop we'll drop again ID we don't need X is is one then I will write U equals to species size = to three then markers equal to O SD yeah cool then PLT do show it's 3:35:01 running yeah so first let me make it to full screen yeah so here pair wise is useful uh when you want to visualize the distribution of the variable or the relationship between the multiple variable separately within subsets of your dat okay so here this blue one is satoa ver color and the virginica okay so these are some uh graph you can say or here you can see SE length Okay so green one is virginica and versy color are almost having same sa length Okay and here are the different types of graphs okay if you don't want to uh let's say if you don't know how to read this graph you can use this graph or you can use this graph either you can use this graph okay that's the power of pair pair wise plots you can say and the SLE width CM Okay setosa no Okay virgin can 3:36:02 vertical again having almost same sample width okay and here as well okay this satoa is in different form okay yeah so we create one small uh pair of uh one small graph then we'll move forward okay I will write here PLT dot so now we are creating box plot figure okay then data set dot drop then again ID comma X should be [Music] one dot box plot okay then figure size I chose this yeah let's run it yeah again see this is the box plot petal L same petal WID SLE L SLE width okay according to width it is showing like from this to this width we have the versical and from 3:37:02 this to this width this length sorry SE length we have that VIN lies here Iris vinica and from here to here Iris SATA lies okay the SE cool and like you can use 3D model you can use different types of charts okay did four and four are enough to read the data set okay so now we will do uh some Cann classification okay we will make prediction and we will see the accuracy of our data set okay so now what I will do I will write here from eSalon Dot neighbors import K neighbors okay WR from Escalon dot neighbors import K 3:38:05 neighor classifier okay then I will write here from skarn Matrix import confusion Matrix comma accuracy accuracy score okay then I will write from scar. model selection import cross value score okay so here what we I did uh we are fitting the classifier to the training set and the loading the libraries basically okay then we'll initiate learning model k equals to three okay so here I will write classifier let me give one space classifier equals 3:39:06 to neighbors classifier and neighbor = to 3 okay then fitting the model I will write here classifier dot fit X train comma y train okay then uh now we will predicting the test uh test set results okay so y prediction equals to classifier do predict X test cool okay from okay spelling mistake okay okay yeah I guess it's fine now so now 3:40:09 let's evaluate the prediction so here I will uh build the confusion Matrix okay so here I will write CM confusion Matrix equals to confusion underscore Matrix uh y test comma y prediction okay then here I will add CM okay so what is confusion Matrix basically so I confusion Matrix uh you know is a table that shows how well a model performs by comparing its prediction to the actual values okay so what it shows a confusion uh M display the number of correct and incorrect prediction of each class in models who either it can give you true positive true negative false positive false negative either it can give you zero or one okay so yeah moving forward let calculate the model accuracy this is the main part if the model accuracy is low it means uh your analysis of you 3:41:10 know classification is not good okay so accuracy it should be more than 80 at least accuracy equals to accuracy underscore score y test comma y prediction into 100 otherwise it will give main points so I will write print model Canon model accuracy is oh I write Plus St Str I will round it okay 3:41:57 accuracy I will add the person so why I wrote this accuracy to round so I don't want after points I need only two numbers okay I don't want like 6 7 8 9 10 11 12 like this okay I need only like 80.20 like this cool let's run this see our KNN model accuracy is 96.67 that's why I wrote two here and the person should be so 96 which is very very very good okay so this is how you can find uh the accuracy so now let's find the optimal number of Neighbors in K okay basically finding the best K so we will use uh using cross validation parameter okay tuning so first I will create the list of K for Kon okay so here I will write Kore list equals to list range 1 comma 50 comma 3:42:58 2 here I'm I will create the list of CV score okay so here I will write CV scores okay okay I have to give the brackets yeah so we'll here perform the 10-fold cross validation I will explain you what is cross validation don't worry first let me write 4K n k list KNN equals to okay neighbor classifier and here I will write and neighbor Neighbors 2 K then SC for to cross value score Cann then X train comma y train okay comma I will write here class 3:44:02 validation equals to 10 scoring equals to I will write here accuracy okay then here CV uncore course. append to sc. me okay so now yeah let me run it okay comma some error came okay the error is the scoring parameter yeah why because here accuracy you can see and I did this spelling mistake okay so now what is cross validation of course cross validation you know determine the accuracy of your machine learning model by partioning the data into two different groups okay called training set and testing set you can see training test and the testing set okay X and Y so the data is randomly separated into a certain number of 3:45:03 groups or subsets called folds okay you can see the 10 folds we have wrote each fold contains about the same amount of okay and there is one more thing validations so validation is a technique for assessing the accuracy of the model on data set okay and this cross validation we did on new data set cool so now let's find the best K okay so here I will write best k equals to K list okay before that I will write one thing I will write here MSE was changing to misclassification error okay so equals to 1 that's X for X in for X in CV scores okay l i will WR msse dot index 3:46:04 minimum MSC I will use this bracket square bracket okay so then I will write print the best as optimal number of neers is person T person bestore K okay let's run it see the best optimal number of neighbors okay neighbors is nine Okay so in the K nearest neighbor Cann algorithms okay so K represent the number of neighbors that are considered when classifying a query Point okay see the if we'll classify this particular point you will get the six okay 1 2 3 4 5 6 okay and uh let me show you if you will classify this 3:47:06 portion only so you will get 1 2 3 4 5 6 points like this okay so the best optimal number of neighbor is nine 3:47:20 [Music] learning objectives welcome to math refresher linear algebra in this lesson we are going to explain the concepts of linear algebra solve a linear system of equations describe Matrix forms of Matrix and Matrix operations and Define vectors and list down on its properties introduction to linear algebra linear algebra linear algebra refers to study of linear combinations for linear transformations to be carried out a study of vector spaces lines and planes as well as some mappings as necessary it contains linear functions vectors and matrices it is an examination of characteristic of linear set transformations linear equations linear algebra's major goal is to establish systematic techniques for solving 3:48:22 systems of linear equations a linear equation with n variables can be represented as A1 X1 plus A2 X2 plus A3 X3 and so on till a n xn equals to B where X1 X2 X3 and so on till xn are the unknown quantities that are to be found while A1 A2 A3 and so on until a in are the coefficients B here is a constant term a linear equation with two variables X and Y is a linear connection between X and Y the value of one variable often Y is determined by the value of the other that is X the independent variable in this situation is X while the term dependent VAR variable refers to Y since it depends on X let's consider some examples for linear 3:49:22 equations linear equations in one variable can be written as 3x + 5 = 0 or 3x 2 of x + 7 = to 0 or 98x = to 49 linear equations in two variables examples can include y + 7x = to 3 or 3 a + 2 B = 5 6 x + 9 y - 12 = to 0 linear equations can also be written with three variables as x + y + z = to Z hey - 3 B = to C 3x + 12 y = to half of Z identifying linear and nonlinear equations linear equations present themselves as a straight line on a graph however nonlinear equations represent themselves as a non-straight line on the 3:50:22 graph here equations like y = to 8x - 9 or y + 3x - 1 = to 0 are examples of linear equations however y = to x² - 7 is a nonlinear equation because X is raised to the^ 2 hence hence denoting a nonlinear relationship or a non-straight line relationship between X and Y similarly Square < TK of y + x = 6 or Y raed to^ 2 - x = 9 are examples of nonlinear equations as Y is raised to the power half and two respectively forms of linear equation linear equations can be represented in three forms a standard form a slope intercept form or a point slope form linear equation in standard form a 3:51:25 linear equation in standard form for a single variable can be represented as a X plus b equals to zero where A and B are real integers and X is the variable while a linear equation in standard form for two variable can be represented as ax + b y equals 2 C where a B and C are real integers while X and Y are the variables to summarize a linear equation for variables can be written as a x + b y = to C where X and Y are the variables C is the constant and a and b are the coefficients of X and Y respectively linear equation in the slope intercept form a linear equation slope can be calculated to see how one variable varies in response to a unit change in another variable it is represented as y = to mx + b where m is 3:52:28 the slope B is The Intercept on the Y AIS and X and Y are the lines distance from the X and the Y AIS respectively consider this example the lines point x y repres represents the distance from the X and the Y AIS respectively while the line intercept on the Y AIS is at 0 comma B linear equation slope the slope indicates how steep a line is with respect to Y AIS when seen from left to right it indicates whether the line goes up or down the slope describes how independent VAR able has been changed while the dependent variable is changing types of slope there are four types of slopes based on the relationship between the two variables X and Y the first is the positive slope 3:53:30 here as X increases Y is also increasing in a negative slope as X increases y decreases if a line is parallel to xais it is a zero slope on the other hand if the line is parallel to Y AIS then the slope is undefined linear equation in point slope form a straight line is represented in point slope form by its slope and a point on the line it is represented as y - y1 = to M of x - X1 where X1 and y1 are the coordinates of the point linear equation forms some examples consider solving the linear equation given as 2x - 10 / 2 = 3x - 3:54:30 1 first we will clear the fraction in this example we can see only left hand side we have 2x - 10 / 2 if from the numerator we take two as common we will be left with 2 * x - 5 / 2 hence x - 5 on the left hand side on the right hand side we still have 3 * x -1 now as a part of next step we simplify both sides of the equation that is open up the brackets when we do that on the right side we end up getting 3x - 3 by multiplying 3 to X - 1 solving further we can write as X = 3x + 2 finally we try to clear the fraction to solve for x we can rearrange the equation as x - 3x = to 2 solving it further we get - 2x = to 3:55:33 2 and then we solve for x as -1 system of linear equation a system of linear equations is a finite collection of linear equations usually involving the same variables it can be represented as a11 X1 + A1 2x2 and so on equals to B1 a21 X1 + A2 2x2 so on equals to B2 and so on here this example is for n variables a system of M equations a linear system can have unlimited number of solutions one solution or no solution at all in case of a single linear equation a consistent linear system has a solution while an inconsistent linear system has no solution solving a linear equation solving linear system of 3:56:34 equations various methods can be used to solve linear system of equations which include graphic method substitution method linear combination or elimination method or a matrix method graphing method let's try and solve these two linear equations using the graphic method the first equation is defined as y equal 0.5x + 2 and the second equation is Y = -2x - 3 the two equations as we can see are in the slope intercept form the first line has a slope of 0.5 and a y intercept of pos2 while the second line has a slope of -2 and a y intercept of -3 let's have a look at these lines through a plot here the blue line is the first line represented by y = 2.5 x + 3:57:38 2 and the red line is represented by y = to -2X 3 these two lines intersect at a point at -2 comma 1 hence we can say that X = to -2 and Y = to 1 solving systems of linear equation using substitution for substitution we start by putting one of the equations in the form of variable equals 2 we then substitute that variable in the other equation in its place take the other equation to solve if there are more than two variables involved we repeat the process from steps 1 through three let's consider an example let's consider the equation 3x + 2 y = to 19 and x + y = to 8 we could 3:58:41 begin with any equation and variable in this case let's look at the second equation with variable y we start with subtracting X from both sides of x + y = 8 and we end up getting y = to 8 - x the first equation still retains its form now we take the value of y that is 8 - x and substitute in the first equation in place of Y we end up getting the equation 3x + 2 * 8 - x = 19 we further simplify and solve this equation using algebra method to get 3x + 16 - 2x = 290 we solve this equation further and we end up getting x + 16 = to 19 and finally X = to 3 now we take the value of x X and 3:59:44 substitute in the new reformed second equation as y = 8 - x that gives us y = 8 - 3 hence y = 5 The Final Answer remains x = 3 and Y = to 5 solving systems of linear equation using elimination in this case we start by multiplying an equation by a constant anything other than zero we then add or subtract an equation onto the other equation let's consider the same example of 3x + 2 y = 29 and x + y = 8 we first start by multiplying the second equation by 2 and we get 2x + 2 y = 216 now we subtract the second equation from the first equation that that is 3x - 4:00:46 2x 2 y - 2 Y and 19 - 60 on solving this we end up getting X = to 3 Now using these steps of substitution we put the value of x in the original second equation that is x + y = 8 and achieve a value of y equal to 5 hence the answer remains X = to 3 and Y = to 5 we can verify the solution by using the graph here again the Blue Line represents the point at which 3x + 2 y = to 19 is true while the red line represents the point where X+ y = to 8 is true the solution is found at the point where X = to 3 and y equal to 5 this is the point where both lines 4:01:49 intersect introduction to matrices Matrix a matrix is rectangular array or table with rows and Columns of numbers symbols Expressions used to represent a mathematical object or an attribute for example here a is a matrix of two rows and three columns as 2 1 3 - 1 2 and 4 well B is a squared Matrix with two rows and two columns given as 2 1 3 4 Matrix size The Matrix size is expressed as M cross n where m is the number of rows and N is the number of columns a matrix with two rows and three columns can be referred to as 2x3 Matrix or a two cross cross 3 Matrix or a matrix of Dimension 2 cross 3 notation of a matrix a matrix with M 4:02:52 rows and N columns can be presented as this here each element will be identified by its row number and column number for example the element over here is in the third row and the First Column hence it is identified as a 31 forms of Matrix The Matrix is termed as a squared Matrix if n equals to M that is the number of rows and number of columns are equal an element's entry in The Matrix of from a i i is located on the diagonal he is called a diagonal matrix if he I J equals to zero when I is not equals to J that is in a square Matrix apart from diagonal at all places 4:03:52 we have zero as a value Matrix operations addition addition of two matrices is adding their corresponding elements as in a11 + b11 a12 + B12 a21 + B21 a22 + B2 2 and so on consider the example over here we have a matrix a as 22 32 11 16 and Matrix b as 13 8 13 and 16 Matrix a is of the order 2 cross 2 and Matrix B is also of the same order 2 cross 2 hence the corresponding elements can be quickly added as 22 + 13 32 + 8 11 + 13 and 16 + 16 giving us the result in Matrix of 35 40 24 and 4:04:53 32 addition rules only matrices with the same size can be added that is the two matrices should have same number of rows and same number of columns Matrix addition follows commutative property that means a + b is equal to B plus plus a subtraction Matrix subtraction is same as Matrix addition it is subtracting the corresponding values let's consider the same example of Matrix A and B the result of a minus B can be given as 22 -3 that is a11 - b11 32 - 8 that is a12 - B1 1 2 11 - 13 a 21 - B21 and 16 - 16 that is a22 minus 4:05:54 B22 giving us result as 9 24 -2 and 0 subtraction rules only matrices with the same number of rows and columns can be subtracted Matrix subtraction does not follow commutative property that means a minus B is not equals to B minus a multiplication matrix multiplication results in what is called as a product Matrix in matrix multiplication each row of the first Matrix is Multiplied to each column of the second Matrix to compute each of these element in the Matrix the values are then added for example consider the Matrix A and B to get the first element we will multiply the first row of Matrix a to 4:06:54 the First Column of Matrix B and then add the individual elements that is 22 multiply to 13 and 32 multiply to 13 and then add them up to get this particular element for the second element we multiply the first row of a to the second column of B hence getting 22 multiplied to 8 + 32 multiplied to 16 similarly we compute all the remaining elements that is the second row multiplied to the First Column and added 11 multiplied to 13 + 16 multiplied to 133 and the second row now multiplied to the second column that is 11 * to 8 + 16 * to 16 this gives us the resulting Matrix as 702 4:07:54 688 351 and 344 multiplication rules let us say C equals to A here C is our product Matrix which can be computed as summation over a at I j. bjk for a matrix of size i j Matrix B of size JK to calculate the value of each member in the I cross K Matrix of C the matrix product AB is defined only when the number of columns in a equals to the number of rows in B the matrix product of ba is defined only when the number of columns in B is equals to the number of rows in a a do B is not always equals to B do a transpose a transpose is a matrix form 4:08:57 by turning all the rows of a given Matrix into the columns and vice versa the transpose of a matrix a is denoted as a for example if the Matrix a is 22 32 11 and 16 the transpose of a would be 22 11 32 and 16 here we can see the row has been converted to column inverse if it is a non- singular Square Matrix there exists n cross n Matrix a minus one known as A's inverse that satisfies a. a inverse equals 2 a inverse. a equals to I where I is the identity Matrix and identity Matrix is characterized as a diagonals one and all other elements are 4:09:58 zero also AB equals to b. a equals to i n where I in denotes the N cross n identity Matrix special Matrix types symmetric Matrix a matrix a is set to be symmetric if a is equals to transpose of a diagonal matrix a matrix D is diagonal only if D equals to zero for all I not equals to J identity Matrix identity Matrix is denoted as I in where a i equals to I a equals to A tensors tensors are arrays with more than two axes a tensor can have in dimensions a i j k is the value in the tensor a at coordinates i j and k the principal 4:11:00 component analysis PCA which is a statistical technique used to reduce the number of dimensions in a given data set while we preserve its variability so there are different use cases of PCA in realtime scenarios one of the major use case is in the healthc care domain so a technically known as the spike triggered covariance analysis it uses the principle of PCA in Neuroscience to identify the specific properties of a stimulus that increase a neuron's probability of generating an action potential so it is using reduction techniques which is going to be really helpful in detecting the coordinated activities of large neuronal ensembles so it is basically used for identifying a neuron from the shape of its action potential then in terms of quantitative Finance so PCA help us in reducing the dimensionality of a complex 4:12:00 problem let's say a fund manager has 200 stocks in his portfolio now to analyze these stocks quantitatively a stock manager will require a correlational metric of a size 200x 200 which makes the problem very complex however if he has to extract 10 principal components which best represents the variance in the stock price this will reduce the complexity of the problem while still explaining the movement of all 200 stocks so we use it for analyzing the shape of the yield curve in terms of forecasting the portfolio returns developing the asset allocation algorithms and so on and it is also widely used in image compression so PCA is also used for image compression so if we go ahead and use any of the pie charm IDE so here we can work with Matt plot lib image P plot and the nump as NP and here we can also import from sklearn and 4:13:03 we can import the PCA so basically we are trying to print the shape of the image by reading the image this is a sample logo. PNG which has been placed in the same directory we are trying to print the image so if we run this particular code then we will be able to see the shape of the image return to us first the current shape based on the image which has been deducted which is 156 94 and 3 so what does this mean this means image is in the form of 156 rows and each row containing one 194 pixels and then having three channels which is red green and blue so we had to resize the image so that it is in a format required for PCA input which is 196 by3 which is going to be 582 so here we can see we have performed a simple operation as image R where we 4:14:06 are using the numpy library for reshaping the image and then we are printing the shape of the image again so here we can see 156 and 582 has been returned as a response now running the PCA with 32 principal components we can use this fit for the image that has been reshaped and then we are trying to train the transform image so with these 32 components we are able to express 98.7% of the variance so inverse transforming the PC output and reshaping the visualization can also be done forward and that's how PCA is widely used as a part of the medical and the other Sciences for example is used for object detection model for the healthcare domain for the transportation Aviation and location-based Services introduction to vectors objects with both magnitude and direction are called as vectors the 4:15:08 magnitude of the vector determines its size it is represented as a line with an arrow where the length of the line indicates the vector's magnitude and the arrow points in the desired Direction other names for it include ukian Vector geometric Vector spatial vector or simply Vector its length is indicated by the symbol double bar vble bar and it begins at the original Z comma 0 notation of a vector a vector can be represented in a standard form as a equal to a I cap plus b j cap plus c k cap where a B and C are numeric values the unit vectors along the XY Zed axis are i j and k cap respectively imagine you are a robot in a maze and every step you take bring you closer to the treasure or into the Trap but here's a catch you don't have a map 4:16:10 so how will you figure out the best path without any guidance this is where Q learning steps in one of the most exciting branches of reinforcement learning it is like teaching an agent or a robot to play a game of trial and error learning to maximize rewards and avoid penalities all without knowing the rules in advance in this tutorial we will dive into the Core Concepts behind Q learning from understanding how an agent explores it environment to making smart decision that leads to Victory now before we move on and learn more about Q learning I request you guys that do not forget to hit the Subscribe button and click the Bell icon for further updates so let's start by understanding first what is Q learning so Q learning is a type of reinforcement learning which is basically used in machine learning algorithms to find Optimal action selection policy for any given problem for reinforcement learning now you'll be wondering what this type of technical jargon means so let me tell you in easy 4:17:12 words what reinforcement learning exactly means and how Q learning is a part of it so let us consider a scenario imagine there is a robot whose task is basically to clean the room a kind of vacuum cleaner robot in that room there will be obstacles like furniture and there are dirt scattered around the goal of this robot is to learn how to clean the room efficiently now there are certain points points so this is called learning by rewards so we have plus one for positive reward if the given vacuum cleaner cleans up the dirt we have minus one or negative reward for bumping into the walls or any Furnitures and we have the higher plus 10 reward for this robot when it completes the cleaning of a room in a shorter period of time now if we talk about this kind of learning how this robot is learning so suppose this 4:18:13 robot starts up to cleaning the room now after it has started to clean the room it found some obstacle and it got collided with this so it learned that it is getting minus one for the same so next time when it encountered any obstacles it just you know ignores it or you know it just changes its direction and this is called learning by rewards so in the scenario we have the agent as robot the environment which we have is room full of dirt Furnitures and wall now if I talk about actions then we can say the robot can move Left Right backward or forward so this is a scenario that we have set up for our robot so by trial and error the robot learns a strategy or a policy that maximizes its reward by avoiding obstacles prioritizing areas with more dirt and move efficiently to cover the entire room eventually after enough learning episodes the robot vacuum develops an optimal cleaning strategy 4:19:15 navigating the room without getting stuck and cleaning it in the shortest time possible but this is the scenario for the reinforcement learning and under this comes the Q learning so in the simpler words if I have to say when Q learning is basically a machine learning approach that enables a model to iteratively learn and improve over time by taking the correct action which means the optimal action selection policy so as we have seen in the example how this robot has maximized its total reward by avoiding obstacles prioritizing areas with more dirt and moving efficiently to cover the entire room this is one of the examples of reinforcement learning so this is basically a q learning approach okay this is a part of machine learning now let's us move ahead and try to understand the components of Q learning now if I talk about Q learning guys so now discover the key components of Q learning the first one that we have all 4:20:17 over here is State action and rewards so basically Q learning revolves around it now let us dive deep into it and understand more of the key components so if you talk in depth about the key components so we have the Q values or the action values we have so if I talk about Q values guys Q values are basically defined for States and action so it is written as Qs of a which is an estimation of how good is it to take an action at a given status and this estimation of Q of s of a will be iteratively computed using the TD update rule or we can say temporal difference update rule which we are going to see in our given example so if I talk about rewards and episodes so we can see all over here that an agent like throughout its lifetime starts from a start State and makes several transition from its current state to the next state based on 4:21:17 its choice of action and also the environment the agent is interacting so at the every step of transition the agent from State takes an action observes a reward from the environment and also then do the transition to the another state so for example if at any point of time the agent ends up in one of the terminating States okay that means like there are no further transitions possible so this is meaning that it is completed one episode so this is rewards and episodes for the same the next that we have all over here is temporal difference so the temporal difference update Rule now if you talk about temporal difference it could be written something like this so you can see an equation all over here so this shows the temporal difference equation where we have q of s OFA and this is being redirected all over here so Q of s OFA plus an alpha r + gamma Q of new values 4:22:17 of sna and minus Q of sna so what is this basically so if you talk all over here this is basically an update rule okay I'll write all over here now this update rule basically what it does guys that it is applied at every time at the step of agent interaction with the environment now you would be wondering what SN AR so s if I talk about s is so if I talk about s s means the current state of the agent and if I talk about a it is according to the current action picked according to the policy now if I talk about the changed s Dash so it just means the next state where the agent ends up 4:23:22 and if I talk about a dash all way here so it means that next best action okay to be picked using the current Q value so these are the certain definitions all over here and uh if I talk about certain other variables all over here so we have also seen in this equation we have R Alpha and Gamma let us discuss about R Alpha and Gamma okay now if I talk about R guys so R here is you know the current reward which is basically observed from the environment in response of the current action okay so if we talk about R guys r means current revolved observed then if I talk about Gamma okay so gamma here means there are two condition either it should be greater 4:24:25 than or equal to zero okay or it can be less than equal to 1 now this is basically a discounting factor this means that gamma is a discounting factor in our given determination of the Q values and this is basically for future rewards so future rewards are less valuable than the current rewards so they must be discounted okay this is a logic for it and uh since Q value is an estimation of expected rewards okay so here from a given state the discounting rule also applies all over here so for the same purpose we'll be using gamma and if I talk about Alpha all over here so so Alpha is a step length basically okay let's remember it Alpha is a step length which is here taken to estimate the update estimation 4:25:27 of qfs of a okay so this is basically a step length how much steps it takes to update you know how much this uh given state of the robot is you know updated basically the step length taken to update the estimation of this factor or this Q of s of a this is the alpha now we have understood the temporal difference how we have calculated using this formulation okay and what are the mathematical variables involved in this step now finally we have exploration and exploitation so basically exploration and exploitation comes under selecting the course of action with a greedy policy which is basically a simple method for selecting an action to take based on the current estimates of the Q value so here the greedy policy applies now if I talk about exploitation all over here it means the superior Q value action so we 4:26:30 have the probability all over here that there will be certain probability and the factor Epsilon all over here it should be less than that now exploitation meanss all over over here that to use its current knowledge and explore means try new things so here the egy policy helps the agent to decide what to do whether to exploit means to use a current knowledge or to explore try new things so let me tell you the mathematical formulation for the same so if I talk about exploitation so here most of the time there's a probability of 1 minus E Okay so it is basically Epsilon and here the agent will pick the action that has currently the highest quick value so if I talk about exploitation guys it means that it is using the current knowledge and most of the time with a probability of 1 minus E or 1us 4:27:33 Epsilon so here the agent will pick the action that currently has the highest Q value and this is called exploitation because the agent is using its current knowledge to choose what it thinks is the best for the action if I talk about exploration all over here it means trying new things so here we have the small probability e okay the same e that we are using all over here now the agent will take up a random action instead of the best known action and this is called as exploration because the agent is trying to discover if there are better option available or not so by balancing the exploitation and exploration the agent can learn better over time if it only exploits it might misdc discovering the better strategies that are available and also if it explores too much it might waste a Time on bad actions so the EG greedy policy okay which helps us to decide what agent should do basically 4:28:34 combine both exploitation and exploration so this is a method which helps the agent to find the right balance between doing what it knows is best and trying new things to improve its knowledge now if I talk about determining Q value then here we have the two techniques but what is Q value exactly Q value is the metrix that is used to evaluate actions at a specific States so as we have discussed in the given example of the robot so there can be a various state for a robot so we have to calculate the metrics which will be helping us to evaluate what action should be taken at the specific States so there are two ways to calculate the Q values the first one is the temporal difference that we have discussed the second one is belman's Ford equation Q of sofa R of s OFA GMA Max Q of s of Dash and a since we have discussed all about it okay now if I talk about 4:29:35 temporal difference it is calculated by comparing the current state and action values with the previous one and belman Ford equation is a recursive equation okay which is basically a formula invented by the scientist belman Ford and it is used to calculate the value of a given State and determine its optimal position it provides also a recursive formula for calculating the value of a given state so in a marov decision process so it's a mathematical term that we use for the stochastic processes and in particularly it is very very influential or you can say very much important in the context of Q learning and optimal decision making so this equation that I have shown you earlier is also very very important in determining the Q now similarly we create the Q table so Q table is a repository of rewards basically which is associated with the optimal actions for each state in a given environment it 4:30:35 basically serves as a guide for the agent which helps it to determine which actions are likely to yield the best outcomes so agent will always be interacting with the Q table and dynamically update the reflected agents in the evolving understanding and also enabling more informed decision making now let us try to implement the Q learning environment so there are certain steps that we have to follow the step one will be defining the Environ so let us type step one that we have defining the environment so here we have have to set up our environment parameters including like number of states action and initialize the Q table so let us mention all over here that we have like number of states okay we have actions and initializing the Q table now 4:31:36 in The Next Step all we have we have to set up the hyper parameters now if I talk about setting up the hyp parameters so with respect to the Q learning algorithm we have to include the learning rate okay we have to also include the discount Factor we have to include the exploration probability we have to also train the number of training impo so this is the parameters that we have involved in the second step now in the step three we have to implement the Q learning algorithm so we'll be performing Q learning algorithm over multiple of epochs so each Epoch will be involving selecting the action based on Epsilon greedy strategy and also updating the Q values based on rewards receipt and also 4:32:37 that will help us in transitioning to the next state now so here be wondering what EPO means okay so in a machine learning algorithm an Epoch is basically a complete pass of a training data which is set through an algorithm so it means that it is a single pass of the entire training data set so the model updates is internal calculation after each Epoch means the entire training data set that we have taken you know to study basically for our given model so that is an epoch single pass so when all the training data is used at once and it is defined as a total number of iterations of all the training data in one cycle for a given training then for that machine learning model it is called an Epoch okay so I hope so you would have got an idea regarding uh epox now let us try to code this in Python so you can see all over here so this is a given code for 4:33:37 our Q learning algorithm so here I'm importing nump as NP okay that is our library that we'll be using then here we are defining the environment okay the first one is number of states okay say n equals to 16 then number of actions so number of possible actions that we have discussed for a given robot so up down left and right and what is the goal State okay this is our environments that we have set up so we have States action okay and goal State now we'll be initializing our Q table with zero so for the same purpose we'll be using using numpy so np. Z and where we have nend States and action so we have initialized our Q table now we are defining the parameters all over here so we have the learning rate at 0.8 discount Factor we have given certain value exploration probability and EPO so I have told you about epox guys so epox here means that when all the training data is used at once and it is defined 4:34:37 as a total number of iterations for all the trading data in one C for a training so that is Epoch and that we have set to 1,000 now for the Q learning so we have taken this Epoch okay so for Epoch in the range of epochs this is iteration so we are setting the current state okay so we are using np. random and it is taking a random integer it is starting from any random State all over here this line means that same so now we are using the conditional so while the current state is not equal to the goal State okay this is our go goal state that we have to achieve okay that is 50 so choose the action with the Epsilon greedy strategy so here we have applied the logic for the same so if np. random. Rand is less than exploration probability then our current action is set to np. random randant and here zero N Action so if I talk about N Action means number of possible actions that can go up which 4:35:38 can be up down left and right so it is is choosing the best action with the Epsilon greedy strategy or else we are putting the action as NP AR Max and now we are updating the Q table with respect to the current state this is the exploitation So based on your Q table choose the next state so this is like selecting you know what should an agent do it should explore or exploit so for exploitation it is using the Q table that we have initialized earlier and for exploration it is using the NP random the Randon and it is choosing action up down left and right so based on the probability of Epsilon the exploration action is also going to happen now we have to simulate the environment or move to the next state so here the next state is set to current state plus one Modelo n States now you can see we have initialized the N States at 16 so number of states available total okay consider 4:36:38 it as a grid So based on that now now here what we have done defining a simple reward function okay so here if the goal state is reached so you will be given one point otherwise the probability will be zero so reward is one if the next state goal equals to goal State otherwise put it up as zero now we have to update the Q value so all over here we have updated the Q value so Q current state and action plus learning rate multiplied by you know reward Plus discount Factor multiplied NP Max and in parenthesis we have q table and the next State minus Q table current state in action so this is basically the formulation which I have told you how the updating Q value happens with the help of belman for's equation okay now I'll write all over here now finally you are moving the current state to the next state and after training you are printing the 4:37:40 Learned Q Val now let us try to run this make sure that your server is connected so you can see we have got the Learned Q table so we have all the possible values all over here okay and in this way the agent is going to select from this Q table okay which action should take So based on the given data that we have given we have successfully you know created our Q table with the help of numai so this was a very simple example to Showcase you how the robots in our present scenario like which are learned robots which can you know tackle the obstacles and also learn you know which step to take next so this is one type of the reinforcement learning and uh I hope so you would have got a brief idea regarding this algorithm hello everyone I am M and welcome to today's video where we will be talking about llm benchmarks tools used to test and measure how well large language models 4:38:41 like GPT and Google Gemini performs if you have ever wondered how AI models are evaluated this video will explain it in simple terms llm benchmarks are used to check how good these models are at tasks like coding answering questions and translating languages or summarizing text these tests use sample data and a specific measurement to see how will the model perform for example the model might be tested with a few example like few short learning or none at all like zero short learning to see how it handles new task so now the question arises why are these benchmarks important they help developers understand where a model is strong and where it needs Improvement they also make it easier to compare different models helping people choose the best one for their needs however llm benchmarks do have some limits they don't always predict how well a model will work in real world situation and sometimes model can overfit meaning they perform well on test data but struggle in Practical use we will also cover how llm leaderboards rank different model 4:39:43 based on their benchmark scores giving us a clear picture of Which models are performing the best so stay tuned as we dive into how llm benchmarks work and why they are so important for advancing AI so without any further Ado let's get started so what are llm benchmarks llm benchmarks are standardized tools used to evaluate the performance of L language models they provide a structure way to test llms on a specific task or question using sample data and predefined metrics to measure their capabilities these Benchmark assess V various skills such as coding Common Sense reasoning and NLP tasks like machine translation question answering and text summarization the importance of llm Benchmark lies in their role in advancing model development they track the progress of an llm offering quantitive insights into where the model performs well and where Improvement is needed this feedback is crucial for guiding the fine-tuning process allowing researchers and developers to enhance model performance additionally benchmarks offers an objective comparison between different llms helping developers and organization 4:40:44 choose the best model for their needs so how llm benchmarks work llm Benchmark follow a clear and systematic process they present a task for llm to complete evaluate it performance using specific matrics and assign a score based on how well the model performs so here is a breakdown of how this process work the first one is setup llm Benchmark come with pre-prepared sample data including coding challenges long documents math problem and real world conversation the task is span various areas like Common Sense reasoning problem solving question answering some regeneration and translation all present to the model at the start of testing the second step is testing the model is tested on one of the three ways few short the llm is provided with a few example before being prompted to complete a task demonstrating its ability to learn from limited data the second one is zero shot the model is asked to perform a task without any prior examples testing its ability to understand New Concept and adapt to unfamiliar scenarios the third 4:41:46 one is fine tune the model is trained on a data set similar to the one used in The Benchmark aiming to enhance its performance on the specific task involved the third step is scoring so after completing the task The Benchmark compares the model's output with the expected answer and generates a score typically ranging from 0 to 100 reflecting how accurately the llm perform so now let's moving forward let's see key metrics for for benchmarking llms so llm Benchmark uses various metrics to assess performance of large language model so here are some commonly used metes the first one is accuracy of precision measure the percentage of correct prediction made by the model the second one is recall also known as sensitivity measure the number of true positive reflecting the currect prediction made by the model the third one is F1 score combines both accuracy and recall into a single metric weighing them equally to address any false positive negatives F1 score ranging from 0 to one where one indicates perfect precision and recall the fourth one is 4:42:47 exact match tracks the percentage of predictions that exactly match the correct answer which is especially used for the task like translation and question answering the fifth one is perplexity gges here it will tell you how well a model predicts the next word or token a lower perplexity score indicates better task comprehension by the model the sixth one is blue bilingual evaluation understood is used for evaluating machine translation by comparing andrs sequence of adjacent text element between the models output and the human produced translation so these quantitative metrics are often combined for more through evaluation so in addition human evaluation introduces qualitatively factors like coherence relevance and semantic meing provide a nuanced assessment however human evalution can be time consuming and subjective making a balance between quantitative and qualitative measures important for comprehensive evaluation so now let's moving forward see some limitation of llm benchmarking while llm benchmarking available for assessing 4:43:47 model performance they have several imitation that prevents them from the fully predicting real world Effectiveness so here are some few the first one is bounded scoring once a model achieves the highest possible scores on The Benchmark that Benchmark loses its utility and must be updated with more challenging tasks to remain a meaningful assessment tool the second one is Broad data set llm Benchmark often rely on Sample data from diverse subject and task so this wide scope may not effectively evaluate a model performance in edge cases specialized fields or specific use cases where more tailored data would be needed the third one is finite assessment Benchmark only test a model current skills and as LMS evolve and a new capabilities emerge new benchmarks must be created to measure these advancement the fourth one is overfitting so if an llm is trained on the same data used for benchmarking it can be lead to overfitting where the model performs well on the test data but struggles with the real task so this result is scores that don't truly represent the model's broader 4:44:48 capabilities so now what are llm leaderboards so llm leaderboards publish a ranking of llms based on the variety of benchmarks leaderboard provide a way to keep track to the myad llms and the compare their performance llm leaderboards are especially beneficial in making decision on Which models to you so here are some so in this you can see here open AI is leading and and GPD 40 second and the Llama third with 45 parameter B and 3.5 Sonet is there so this is best in multitask reasoning what about the best in coding so here open AI o1 is leading I guess this is the orian one and the second one is 3.5 Sonet and after that in the third position there is GPD 4 so this is in best in coding so next comes fastest and most affordable models so fastest models are llama 8B parameter 8B parameter and the second one is L ma Lama 70b and the third one is 1.5 flesh this is Gemini one and 4:45:52 lowest latency and here it is leading Lama again in cheapest models again Lama 8B is leading and in the second number we have Gemini flash 1.5 and in third we have GPT 40 mini moving forward let's see standard benchmarks between CLA 3 Opus and gb4 so in journal they are equal in reasoning CLA 3 Opus is leading and in coding gp4 is leading in math again gp4 is leading in tool use cloud 3 opas is leading and in multilingual Cloud 3 Opus is leading in this tutorial we will learn how to run llm locally in the Windows command prompt but before that let's take a look at llm llm or large language models are Advanced AI systems trained on vast amounts of data enabling them to generate humanik responses and assist in various tasks like coding content creation and automation the power of 4:46:55 llms lies in their ability to understand context generate coherent text and adapt to different use cases making them indispensable in AI driven applications traditionally running llms required cloud-based Solutions but with tools like olama users can now run llms locally on their own devices AMA simplifies model deployment offering privacy low latency and cost efficiency by eliminating the need for external service this local execution empowers developers and businesses to integrate AI capabilities seamlessly without relying on the third party cloud services that's it if these are the type of videos you'd like to watch then hit that like And subscribe buttons and the bell icon to get notified so as discussed to run any launch language model locally you will need the help of AMA so let's get started with one of the browsers and then navigate to AMA 4:47:56 website and start downloading it now we are on Google so let's just navigate to AMA firstly you need to download but before you can see there are a lot of llms here deep SE P mral right so you can see there are a lot of llm options to down download let's download AMA first and here you have mac Linux and windows so we're working on windows so let's go with Windows you can see the download will start shortly there you go it's started so it's about 769 MP so it might take a little bit of time let's wait so far there you go the setup has been successfully downloaded now we need to install it we are now in download section there you go start the application to start installing procedures as simple as that just click on install and agree to some agreements so yeah I think some other okay I think we have clicked on that set of file multiple times so multiple installation procedures are going on don't worry you can just cancel all the other ones slowly there you go the extraction 4:48:58 process has been begun it might take a couple of time there you go so whenever an extra AMA installation process is started you just cancel it this is happening because of a slight lag in the PC but don't worry about it and just cancel and the initial process will continue as it is so the installation is close to be done now we have a lot of variant of llms on olama you can go with R1 you can go with GPD you can go with llama and different varieties of it right and there's also a link to ol GI file where you have all these models and also all the same models are also available in official website as well there you go now we on the OAS website let's close the command prom it's not now all you need to do just add the command line there so if you go into models you have a lot of models there like deep seek R1 llama Fe right and if you go to that 4:49:59 particular if let me expand it okay it got a little more expanded than usual don't worry so here you have 1.5b 7B 8B 14b so what are all these right now let's go to the GitHub for a clearer picture so here you have it 7B 671b but for the same BC carbon so what is it and even the size 4.7 GP and 400 GP right so b stands for parameters billions of parameters 7 billion parameters 6 71 billion parameters and accordingly the size right so based on the space available in your system choose your llm since we have uh constraints on Space you can go with the lower parameter version like I'm going for the first one which is 7b1 and you have a lot of other models as well so to run this particular deeps R1 with 7 billion parameters of size 4.7 B you 4:51:00 just need to copy that single code here which says AMA Run Deep seek R1 correct so you just need to copy that open the command prompt and here you'll just paste that code and execute it let's expand the screen paste it here and now hit enter now the manifestation process right good name they're pulling the file basically but they' have named it as pulling manifest nice and this might take a little while because we are doing it for the first time initially based on the size of the file you have chosen it might take a couple of hours or as well as couple of days in few scenarios if you're going with the biggest one 400 GB version it might take a day or two basically it'll show you 24 hours but it'll take more than that sometimes but now we're going with the lower one which is about 7 billion parameters in 4GB file so it might take little bit time comparatively less to the previous one now let's kind of uh wait for a while there you go it's close to be done 4:52:02 4.6 GB so far and 4.7 that's it a few other files should be running in the background there you go and and shortly we will be ready with the llm which is deep seek R1 similarly we will also go ahead and try with other llms as well first let's begin with deep seek R1 it's verifying a few things and shortly we will be ready with deeps there you go writing manifest success now we will be having the prompt ready yeah there you go now you can just type in high or you can also straight away go ahead with your queries now let's try a simple hi there you go when you run deep seek R1 on browser you can see something called as thinking right so you can see that option here as well when you write some query it will go ahead into the think mode and it will give its thought process to you right now I want to go ahead with a query I want to learn about 4:53:02 let's say large language models but this might take a little while can you help me it's also good if you write it as can you teach me now it started its thinking process I guess we already did a previous version on neural networks based on that let's see what it throws out all right the user is asking if I can teach them about large language models yeah so it remembers the previous queries as well I already provided a general introduction into uh llms in the previous version but anyways it also tries to Deep dive and understand my intention that's the great thing that's exactly what is thinking right now they're deep diving into this particular concept let's wait for a while before the thinking process is done and it comes up with an answer there you go that was its thinking process and also 4:54:04 the end of think so this is the uh thought process which uh deep C carbon went through and it also started to give me an answer of course large language models or llms it started to give me an answer so to learn about llms let's get start with some of the basic concepts so it is starting with the very Basics what is an llm and so far it also gave me how do llms work training models and uh key components of llms it also gave the outputs on applications of llms hyperparameters limitations of llms right and it also gave a few more things reproducibility and fine-tuning if you'd like to Deep dive into any specific aspect of llms feel free to ask there you go that was it and it also concluding some of the critical points in the end that's great let me know how I can help you further now I'll just type by to end the prompt and we can also try 4:55:04 out some different llms from AMA you can just type SLB to end the this particular prom there you go we are done now let's navigate back to olama and since we already tried out with deep seek R1 let's choose another similar llm which is mestral with the same 7.1 billion parameters and 4.5 GB of file size now we'll copy that code and run it over here runal so that we can begin the file or begin the pulling manif test or pull the file and load it into the local browser or local command prompt and uh it will be ready for execution as well so even this will take a little bit of time even though it is just as small as DC car 1 let's wait for a while there you go we almost close to be done 3.94 15 seconds more 10 seconds more and 4:56:06 almost done 4.1 GB and shortly everything will be under extraction process and mistl will be ready for execution unlike the uh neural networks or unlike the complicated questions like llms let's ask mistl a simple question which will be about um solar system so it is verifying a couple of things there you go we have a successful manifest done or the installation is successfully done so let's St in this query that I had in mind hi can you tell me about planetary systems and there you go it started with the thinking process and it'll give you the output shortly there you go hello I'd be happy to share some information about the solar system and it started to give me an output so there you go you can see it started off with Mercury so it also considers the order so you can see Mercury Venus Earth Moon Mars Jupiter Saturn okay it is also 4:57:09 mentioning about the satellite of Earth it's good as long as it is not considering Moon as a planet next we have Neptune the farthest planet now I want to know if it goes with Pluto or not or what it has to say about Pluto that would be the next important question so far we are not considering Pluto as a planet altoe let's see the learning of mist if it considers Pluto as a planet or not let's wait for a while now there you go the Pluto comes into picture I guess yeah the three dwarf are there you go the Pluto okay it is considering it as a dwarf planet and originally consider the N okay it knows that's good there you go I think it ended the solar system knowledge or information that it wanted to share with us there are also many asteroids Commons and small solar system objects if you want to know you can that's it now now let's type in 4:58:11 thanks no let's not it'll again go into the thing mod let's type it as by back slash by to end it there you go now you can also do a couple of more things like AMA list to understand all the listings of llms in your PC or local system so far we have mistl and DEC and if you type in ama it'll start the providing you all the options that you can do with the llm in your or olama in your PC to stop certain model to run a certain model or you can also check the list and you can also remove a certain model if you want further help you can also do that so these are the couple of things that you can do with AMA in your local system and welcome to this video on stable diffusion one of the most advanced AI tools for generating stunning photo realistic images from just text whether you are describing a vibrant Sunset a futuristic city or a sural decape stable diffusion can turn 4:59:11 your imagination into reality within seconds the latest version stable diffusion XEL brings even higher quality results thanks to a larger Network and improved techniques not only you can generate images but you can also enhance them with features like in painting where you can edit parts of an image or out painting which expand image Beyond its original borders so how does it works the AI starts by breaking down an image into noise then cleverly reverse that process C to recreate a clear and detailed picture we will also show you how to create effective promts to get the best result from stable diffusion whether you're using web based version or running it on your own computer and yes you can even use it for commercial purposes stick around because I will be giving you a live demo and showing you step by step how to create your own images with this powerful tool so without any further Ado let's get started so hello guys welcome back to the demo part of the stable diffusion so 5:00:12 first we will I will open stgt AI okay so this is the artificial intelligence company which launched stable diffusion text to image generator okay so we have multiple models in this okay so we have image model we have video model audio 3D languages okay let's go to this image models yes so we have two series sd3 series and sdxl series sd3 large is there t large turbo is there sd3 medium is there in SD XL see stable diffusion XEL is there SD XL turbo is there and Japanese stable diffusion XEL is there so there are two ways of using stable diffusion the first is you can install stable diffusion locally and you can use use it but there are some requirements you should have on 5:01:14 your system okay just like there should be you know uh GPU should be there graphic card and viia graphic card should be there or another graphic card will be fine right so here you can uh you know use the API or you can get the license or you can download the code okay or you can read about the stable diffusion Excel here right so I will show you how to download and install this stable diffuser for a while I don't have any graphic card on my system okay so I can't use but I will show you properly how to use and how to run it so first step I will give you this link okay this is the hugging phase so here sdxl is you know they recently launched model okay so here you can uh read it 5:02:14 about all the configuration or you want okay the how you can install so first you have to install python okay first you have to install python latest version and then second you have to install after installing python you have to install git not get copilot just get Bash okay this you have to install Windows either Mac OS or Linux okay so here what you have to do I will give you this link and you have to go to files and version and you have to download this one sdxl base 1.0 0. this one or you have to download it so see I have already downloaded it okay so this file is all around 6. 5gb okay for a while I'm canceling it because as you can see here I already 5:03:15 downloaded okay so after downloading this you have to go to you know you have to write stable web UI stable diffusion web UI okay I will give you this link as well okay here what you have to do you have to go here and download C file okay again zip file is there but again let me download it for you okay this is done okay after that what you have to do you have to unzip this stable diffusion okay then this folder will come okay then you have to go down and here you can see web U UI user for Windows badge file you have to run this badge file okay if you're you Mac User you can uh run this shell script okay so I 5:04:17 will install this I have already installed but you have to just double click this and everything will be installed okay so it will launch one page after installing this it will take like half an hour because it will download multiple files of 4455 GB okay for a while let me run it see this page will come so as you can see here I am running stable diffusion locally 1 127.0 this okay this is local so here so now we downloaded first file right if you remember we downloaded this file so what you have to do you have to go to downloads then you have to copy this file downloaded file then you go have to go here here you have to go to models then here you have to disable diffusion then you can copy paste here your file okay so these are the models 5:05:21 so this is the latest models okay it will install with v15 prune model but we want to use latest model okay so that's why we are copying it okay so while installing it will show you some these type of things here it will download multiple things right so after installing this just refresh and you can see two models are there okay you can select either okay I'm selecting this here I will write astronaut riding horse but it will give me error okay I will generate so what error is coming see found no and driver on your system okay I don't have any drivers installed in my system I don't have any graphic card but if you have graphic card it will run smoothly it will give you all the outputs but again 5:06:24 we can use this stable diffusion online as well web okay here see stable diffusion 2. one demo I will give you this link as well okay so let me write again then the same thing as not riding horse generate image so it will take 11 second as it is showing we have to wait okay it's scanning almost done processing see astronaut riding horse okay okay let's go to chat gbt and ask some funny give me some funny text to image generators prompts okay so let's get okay A C 5:07:28 wearing chef at adapting oh this one is cool okay let's copy this and paste it here okay let's read a penguin dressed as a pirate searching for treasure on a ice flow with a parrot that only sarks okay see pirate head see pirate head okay let's try something else a robot trying to blend at a grumpy beer sitting in a therapist office okay this one is cool I guess okay let me run it again it will take some 11 seconds but this you know if you run locally this will work definitely I'm sure okay because in other my system it's running 5:08:29 smoothly see a grumpy we are sitting in a therapist office discussing it's feeling about something see sitting this one so this is how you can use use staple diffusion okay locally and using web right this is better than Del because Dal is again expensive hello everyone I am M and Welcome to our YouTube channel today we are exploring hugging fist an amazing tool that makes working with language and AI super easy if you're curious about using advanced technology to understand or create TX this video is perfect for you hugging phas is a company that helps people people use AI models for language tasks like translation text analysis or even generating new text they have created a library called Transformer which comes with pre-trained models so you don't have to build everything from scratch it's simple powerful and perfect for developers researchers or even beginners so in this video I will show you three 5:09:30 cool things you can do with hugging face spe to text turn spoken words into written text easily great for captions or voice apps second one is sentiment analysis find out if a text is positive negative or neutral helpful for understanding reviews or comments the third one is text generation create humanik text that sounds natural perfect for chatbots or creative writing I will also explain some important Basics like how pipelines make using hugging pH model super easy and how tokenization helps AI understand text by end of this video you will know how to use hugging pH to start building your own project projects it's simple fun and really powerful so without any further Ado let's get started so welcome to this demo part of this video so here as you know we will be doing three things first thing speech to text recognition the second thing text generation from a particular sentence or a word and the 5:10:30 third thing we will do sentiment analysis okay so we will perform this one by one so let's start with something cool is piece to text recognition Okay using hugging face so first I will write rename this I write here hugging face speech to text okay yeah so first I will install Transformers Library it is already installed on my system but again just for us I'm doing this okay VIP install Transformer so now let's see what is Transformers so Transformers is a powerful python Library created by hugging face so that allows you to download manipulate and run thousands of pre-trained Open Source AI model fine so as you can see requirement already satisfied it means 5:11:31 like already installed so these Transformer models cover multiple task across you know modalities like NLP natural language processing computer Vision Audio and multimodel learning like many things fine so now I will do from Transformers import pipeline okay so now let's run it so now what is pipeline okay so a Transformer pipeline describes the flow of data from origin system to destination system fine because you already know Transformer means you can run or manipulate thousand of retained open source AI model so pipeline describes the flow of data from origin system to destination system and defines how to transform you the data along the way okay so let's check 5:12:32 the versions transform vers 5:12:49 version okay that long tab okay why is coming import Transformers yeah so we have currently 4.4 2.4 version of the Transformer okay so I'm using Google collab okay so you can use either jupyter notebook visual code Studio or collab okay so now let's import libraries import librosa okay so what is librosa so librosa is a python package for audio and music analysis right because we are doing spe to text so we need this so it provides various functions to quickly extract key audio features and metritic from the audio files okay and this librosa can also be used to analyze and manipulate audio 5:13:51 files in a variety of formats such as wav we MP3 m4a and like that okay so next we will import torch I hope everyone know torch so torch is nothing P torch it is a machine learning library based on the torch Library used for applications such as computer vision and NLP and originally it was developed by The Meta Ai and now the part of Linux Foundation you know umbrella let's import one more thing import so now let's import I python okay Capital IPython do display as display so now you will be wondering 5:14:52 what is this IPython display okay so it is an interactive command line uh for the terminal for Python and it you know it can provide a IPython terminal and the web based notebook platform for python Computing and this uh IPython have you know more advanced features than the python standard interpretor and we quickly execute a single line of python code nothing okay for that I'm using this and the next is from Transformers import wav to vectorizer for CTC and W A 2 vectorizer to 5:15:55 tokenizer okay so now what is this so W2 vectorizer for CTC is supported by the notebook on how to fine tune a speech recognition model okay and this tokenizer so tokenizer is nothing tokenization is a conversion of a text into meaningful lexical tokens belonging to categories defined Lexar program in case of NLP so those categories include nouns verbs adjective punctuation ATC okay now and the last Library let's import numai s NP I hope everyone know numai what is numai again numai is a python Library used for working with arrays it ALS it also has a function you know for working in domain like linear algebra fora transform and matrices and many others fine I hope everyone know about 5:16:57 these Library which we have imported okay so let's move forward WR tokenizer 5:17:17 to to tokenizer Dot from pre-trained this is why I love this Google collab you deite just some words and it will show the suggestions you know Facebook so this is a again one pre uh train model okay so we are just importing it base 96 this is nothing the name okay then model equals to W2 for CTC do from retrained then Facebook W V2 5:18:18 vectorizer two then base fine now let's run it so now you can see so we are loading this okay there the downloading okay so you can ignore these warning so now let's load the audio file so here I will write audio on my sampling trate equal to liosa dot load okay so you you know about liosa right then I have V do M4 a so we I have already one speech or one audio you can 5:19:21 say so I will play it don't worry before the final output I will you know show you 16,000 okay so now I will write okay okay I've already loaded I guess okay okay no issues I will load again okay file is loaded I will name it to V yeah fine so now it won't there and will come again we have v. MP4 okay so 5:20:23 now we have to do okay let me try it we have we. MP4 there but I don't know why it is not [Music] coming okay let me copy the path now it will run I guess it's running yeah so now I will write here audio comma sampling rate okay now listen carefully then I will write display display do audio then part 5:21:26 comma comma autop play equals to True okay hello and welcome this is an AI voice message I guess you heard but again let me play it again hello and welcome this is an AI voice message okay so it is saying hello and welcome this is an AI voice message fine so now what I will do I will input some values input values equals to tokenizer then audio comma return tensors equals to PT find dot 5:22:27 input values okay then here input values okay input values okay yeah so now logits will come here so we have to now store the logits which means non-normalized predictions okay so logits equals to model input values do logits okay then here I write logits okay it's running done so now what we will do we will store predicted IDs then we will pass the logits to values to soft Max to get the predictive 5:23:30 value okay so here I will write predicted IDs equals to torch do ARG Max okay then Logics comma Dimension = to minus one okay then I will pass the you know prediction uh to the tokenizer decode to the transcription okay so here I will write transcriptions equals to tokenizer tokenizer dot decode predicted IDs Zero running fine so now let's see our 5:24:34 output transcriptions so now you can see hello and welcome this is an AI voice message so now let's play it again hello and welcome this is an AI voice message amazing right so this is how you can use hugging face for this piece to text recogition it's a very small code line of code as you can see okay so now let's move forward and do the sentiment analysis okay or text generation so let me open new drive so yeah so now I will write here I will write sentiment slash text [Music] generation hugging pH okay so first of all let's import the 5:25:36 files so First I write [Music] import import warnings then warnings dot filter warnings okay I will write here ignore then we'll import numpy you already know what is numpy and import P so these are some basic python library and I hope everyone know and import M plotly for the plots M plot lip dot pip blot as PLT then I will import cbor for the crafts just static okay SNS then SNS do set 5:26:38 so now we'll import SK learn model train split so SK learn is nothing psychic learn is a probably the most useful library for machine learning python so so this uh Library contains a lot of efficient tools for machine learning and you know statical modeling including classification you can do regression you can do clustering you can do from this Escalon from Escalon dot model [Music] selection import import train test split from Escalon dot matrix 5:27:39 import F1 score comma confusion Matrix comma R Au C or this score so again we'll import teams from Transformers import pipeline everyone know what is this then import torch okay now let's run it you have too many active session 10 minute existing to continue if you're interested in using one more session okay wait yeah fine now which one is open bro terminate 5:28:40 this terminate this terminate this okay fine so now let's do sentiment analysis so uh we will explore sentiment analysis using a pre-trained transform model OKAY the hugging phase Library provides you know convenient pipeline as you already know function that allows us to easily perform sentiment on the text so now let's must import the necessary dependencies and create a sentiments pipeline using this line so I will write here classifier equals to pipeline sentiment analysis then type classifier okay so it will download the pre-trained 5:29:43 sentiment analysis pipeline pipeline is not defined how Okay Okay error is there I hope it will run now yeah so now we can uh pass a single sentence or a list of sentence the classifier and now and get the predictive sentiment labels and Associate confidence score fine so now just for the testing classifier so let's write this is a great movie okay now let me run it so now you can see here label positive so label positive typically refers to the outcome or a class of interest that the model is 5:30:45 designed to predict so here we are just checking the sentiment analysis model okay so this is why I wrote this okay so let's check one 5:31:06 more this was a great course then I did not understood any of it now let's check this yeah perfect and the score you can see the accuracy score 99 % 99% which is almost close to 100% and which is amazing so now you have access to a GPU so you can also utilize it for the faster processing by specifying the device using the parameter okay so now what I will do first I will okay wait yeah fine so now I will 5:32:07 Port data set airine tweets equals to PD do read PD means Panda's Library here we are using okay now first import I will import here itself so I have twitter. tweets. CSV okay tweets do CSV let it upload yeah done so now let me Airline tws do head why doad means you it will show me the top five rows okay 0 1 2 3 4 5 okay you can see TW D line sentiment neutral positive 5:33:09 neutral negative then this this this okay so now let's do something okay so what I will do DF equal DF means data frame [Music] Airline tws then I will write a line sentiment we have Air Line sentiment this column okay comma text I need these two columns okay DF do head five so yeah a ctim neutral and this is what this text is all about okay because these two are the main things text and the sentiment so now let's make plot count 5:34:11 plot then I will write DF comma x equals to Airline [Music] sentiment then palet equals to this then here I will write PLT dox label airline sentiment okay then PLT do viable count PLT do show so it will draw one graph okay okay spelling is 5:35:14 wrong yeah it will show neutral positive sentiment and the negative so as you can see the negative sentences or the sentiments are the more okay so now so we have now three classes which do not match the two class available in the hugging phas pipeline so therefore we will filter out all the rows which have been lab world as neutral okay so DF = to DF and again DF Airline sentiment fine was not equals to neutral okay then DF Target to DF Airline 5:36:15 sentiment dot uh map then I will write here positive positive 1 and thetive 0 fine negative will be Z Z okay then print number of rows comma DF dot shape okay so now you can see number of rows are 11,541 okay okay so now I will write predictions 5:37:16 f it will okay predictions is not defined okay so now what I will do so here I will write Tex to DF text and to list do to list then here I will WR predictions equals to classifier text okay so here write probabilities equals to predictions then for if predictions label 5:38:17 dot starts with starts with p means positive okay else 1 minus prediction score prediction score okay then I will write for prediction and prediction why it is not running this is taking time too much time I don't know why so as you can see finally we have the output of predictions values so this depends on you know system system my system took almost 17 minutes 47 seconds to complete okay so now let's run it okay predictions is not defined Pro 5:39:20 now predictions yeah so now I will write here predictions equals to NP do array NP means do array then I will write one if prediction label is positive dot starts with p okay else zero for prediction and prediction values yep okay fine so now let's check the accuracy of our model print 5:40:22 accuracy okay then I will do round up the values so here I will write NP do mean then DF then the Target right yeah then I will write equal equals to predictions then I will write into 100 comma two then I will write percentage okay fine looks good yes so as you can see our accuracy is 88.9% which is you know very good again so now let's do some confusion metrics 5:41:25 confusion metrix then DF Target comma predictions Comm on normalize to True okay then I will plot confusion Matrix okay DF then Target comma predictions comma normalize equals to True okay okay my bet my bet my bet let here 5:42:28 confusion [Music] Matrix comma labels okay so here uh I will you know plot a confusion mric using cbon okay so here I will use args which is confusion Matrix NP do an array so which is you know labels list so I will WR PLT do figure then figure size to 8 comma 6 that's SNS dot set font scale equals to 1.4 okay then let's create the heat map Fusion 5:43:30 Matrix comma and not will be true okay then I will write fmt = to G then confusion map equals to Blues then x s labels to labels then y t labels equals to labels okay yes so now I will write PLT dot title will be confusion Matrix then PLT dox label will 5:44:33 be predicted values then PLT do y label will be actual values then PLT do show okay why the chart is not coming okay so I have to write plot confusion Matrix then I write cm 5:45:14 negative comma positive okay now the chart will come okay yeah so now you can see here this is you know and this is a confusion Matrix and the negative and the positive ratio is there okay so now let's print the or now let's check the rooc Au score so I will write here print r r score then here I will write r a score than DF Target 5:46:16 comma props okay so 94 okay so first let me tell you what is this Roc AC score so this is the area under the ROC curve so what it does it sum up how well a model can produce relative scores to discriminate between positive and the negative instance across all the classification threshold so with this Roc score of 94 0.94 which is 94% we can conclude that the that a pre-trained sentiment analysis model has achieved the high level of accuracy and Effectiveness so in predicting the sentiment labels so this indicates that the model is capable of accuracy classifying text into positive or the negative sentiment categories Okay so now we'll do text generation okay so text generation involves generating creative and coherent text based thing so I will write here 5:47:18 text okay so text generation involes generating creative and coherent text based on a given prompt or starting point so what we will do first we will import the necessary dependencies and load the data set of the poem okay so here I'm write poems let me write po equals to pd. read. CSV don't worry I will give you these files okay description box below I will add so here I'm WR robot Frost do CSC fine and then I will write poems do head five or head just have to write it module pandas has okay not 5:48:20 doore CSV yeah okay it will show the top five stopping by WS on a Snowy Evening fire and I the aim was Song collection content year of publish okay so now what we will do we will write content equals to 5:48:54 poems content dot drop na to to list okay so to generate text we extract individual lines from the poems and use the pipeline text generation function to create a text generation pipeline fine so here I will write lines equals to then I will write for poem in content okay for line and poem dot split into the next line okay then lines do append then I will add line dot Right 5:49:57 strip to the right it will add okay fine so now lines equals to line for line and lines if length of [Music] line zero then show me the lines five okay so now you have see in here whose fots these are I think I know then the next line his house is in the village though that's next line he will not see me okay like this so now let's you know import that pipeline text generation module okay gen equals to [Music] 5:50:59 pipeline so these are the pre-train model you already know some ation okay so now I will write here lines let's run it yeah done so in line zero you have we have whose votes these are I think I know okay so now we can now generate Tex by providing a prompt and specifying the par such as max length and num return sequence okay why because we have imported this Tex generation module okay for example see gen gen okay sorry lines Z dot max 5:52:00 length max length equals to 20 so now it will generate the this to the maximum 20 okay tell 20 word maximum length okay check okay okay gen expression cannot contain perhaps equals to where okay wait now it will Le so see the line was this much only whose words these are I think I know whose vot these are think I know but here the generated text I wish to go to church because I feel like okay this is how you can do you know text generation now let's check for the more CH lines one okay then max length equals to 30 5:53:03 comma num return sequence say sequences equals to two Okay so our first line was this his house is in the village though okay 0 1 2 3 4 5 okay so here you can see see his house in the village though however you might say that the place was the same with the place this this this this so these are the generated text okay and this is another you know return sequence second so there are two one and the second okay so now let me you know import text St okay then creating function r x I don't need this 5:54:08 then return text St dot fill X comma replace white space white space equals to false false comma fix sentence endings equals to True okay so now out equals to generated lines zero to maximum length equals to 5:55:09 30 then print that out to zero [Music] then [Music] generated text okay so now we are setting uh the P tokens to OS tokens okay so who who these are I think I know and this is the maximum 30 till 30 okay so now I will write here okay preview equals to okay so now what you can do you can you know generate a prompt to generate text on a specific topic like this prompt equals to Transformers have a wide 5:56:10 variety of applications application in NLP okay so this is my prompt okay this I'm not importing from the data set okay so I will write out equals to gen prompt comma max length = to 100 print zap out okay here I will write anyways here I will write generated text 5:57:12 prompt okay so it is running let's wait yeah so okay somewh is there 5:57:45 okay so here the issue is with 100 words I go we go till 50 yeah so let me do it again 100 yeah so now you can see we have we can generate tax using by giving or prompt so we have covered you know three topics in this first thing is piece to text using hugging phase second thing is sentiment analysis using hugging phase and the third is text generation using hugging phase okay and in this Tex generation we have we did with two methods one with the data set and second thing with the you know by giving prompt like in chity you can see so what is deep learning deep learning is is a subset of machine learning which itself is a branch of artificial intelligence unlike traditional machine learning models which require manual feature extraction deep learning models 5:58:45 automatically discovers representation from raw data so this is made possible through neural networks particularly deep neural network which consist of multiple layers of interconnected nodes so these neural network are inspired by the structure and the function of human brain each layer in the network transform the input data into more abstract and comp opposite representation for instance in image recognition the initial layer might detect simple features like edges and textures while the deeper layer recognizes more complex structure like shapes and objects so one of the key advantage of deep learning is its ability to handle large amount of unstructured data such as images audios and text making it extremely powerful for various application so stay tuned as we delve deeper into how these neural networks are trained the types of deep learning models and some exciting application that are shaping our future types of deep learning deep learning AI can be applied supervised unsupervised and reinforcement machine learning using 5:59:47 various methods for each the first one supervised machine learning in supervised learning the neural network learns to make prediction or classify that data using label data sets both input features and Target variables are provided and the network learns by minimizing the error between its prediction and the actual targets a process called back propagation CNN and RNN are the common deep learning algorithms used for tasks like image classification sentiment analysis and language translation the second one unsupervised machine learning in unsupervised machine learning the neural network discovers patterns or cluster in unlabelled data sets without Target variables it identifies hidden pattern or relationship within the data algorithms like Auto encoders and generative models are used for tasks such as clustering dimensionality reduction and anomaly detection the third one reinforcement machine learning in this an agent learns to make decision in an environment to maximize a reward signal the agent takes action observes 6:00:49 the records and learns policies to maximize cumulative rewards over time deep reinforement learning algorithms like deep Q networks and deep deterministic poly gradient are used for tasks such as Robotics and gameplay moving forward let's see what are the artificial neural networks artificial neural networks Ann inspired by the structure and the function of human neurons consist of interconnected layers of artificial neurals or units the input layer receives data from the external resources and it passes to one or more hidden layers each neuron in these layers computes a weed sum of inputs and transfers the result to the next layer during training the weight of these connection are adjusted to optimize the Network's performance a fully connected artificial neural network includes an input layer or more hidden layers and an output layer each neuron in a hidden layer receives input from the previous layer and sends its output to the next layer so this process continues until 6:01:50 the final output layer produces the network response so moving forward let's see types of neural networks so deep learning models can automatically learn feature from data making them ideal to tasks like image recognition speech recognition and natural language processing so the most common architecture and deep learnings are the first one feed foral neural network fnn so these are the simplest type of neural network where information flows linearly from the input to the output they are widely used for tasks such as image classification speech recognition and natural diagnos processing NLP the second one convolutional neural network designed specifically for image and video recognition CNN automatically learn feature from images making them IDE for image classification object detection and image segmentation the third one recurrent neural networks RNN are specialized for processing sequential data time series and natural language they maintain and internal state to capture information from 6:02:51 previous input making them suitable for task such as spe recognition NLP and language translation so now let's move forward and see some deep learning application the first one is autonomous vehicle deep learning is changing the development of self-driving car algorithms like CNS process data from sensors and cameras to detect object recognize traffic signs and make driving decision in real time enhancing safety and efficiency on the road the second one is Healthcare diagnostic deep learning models are being used to analyze medical images such as x-rays MRIs and CT scans with high accuracy they help in early detection and diagnosis of diseases like cancer improving treatment outcomes and saving lives the third one is anal recent advancement in NLP powered by Deep learning models like Transformer chat GPD have led to more sophisticated and humanik text generation translation and sentiment analysis so application include virtual assistant chatbots and automated customer service the fourth 6:03:53 one def fake technology so deep learning techniques are used to create highly realistic synthetic media known as defix while this technology has entertainment and creative application it ALS also raises ethical concern regarding misinformation and digital manipulation the fifth one predictive maintenance in Industries like manufacturing and an aviation deep learning models predict equipment failures before they occur by analyzing sensor data their proactive approach reduces downtime lowers maintenance cost and improves operational efficiency so now let's move forward and see some advantages and disadvantages of deep learning so first one is high computational requirements so deep learning requires significant data and computational resources for training whereas Advantage is high accuracy achieves a state-of-the-art performance in tasks like image recognition and natural language processing whereas deep learning needs large label data sets often require extensive label data set for training 6:04:54 which can be costly and time consuming together so second advantage of deep learning is automated feature engineering automatically discovers and learn relevant features from data without manual interv the third disadvantage is overfitting so deep planning can overfit to training data leading to poor performance on new unseen data whereas the third deep learning Advantage is scalability so deep learning can handle large complex data set and learn from massive amount of data so in conclusion deep learning is a transformative leap in AI mimicking human neural networks it has change healthare Finance autonomous vehicles and NLP hello and welcome to this Tut tutorial on deep learning my name is moan and in the next about 1 1 and a half hours I will take you through what is deep learning and into tensorflow environment to show you an example of deep learning now there are several applications of deep learning really very interesting and Innovative applications and one of them is 6:05:56 identifying the geographic location based on a picture and how does this work the way it works is pretty much we train an artificial neural network with millions of images which are tagged their geolocation is tagged and then when we feed a new picture it will be able to identify the geolocation of this new image for example you have all these images especially with maybe some significant monuments or or U significant locations and you train with millions of such images and then when you feed another image it need not be exactly one of those that you have Tred it can be completely different that is the whole idea of training it will be able to recognize for example that this is a picture from Paris because it is able to recognize the eiel tar so the way it works internally if we have to look a little bit under the H is these images are nothing but this is digital 6:06:57 information in the form of pixels so each image could be a certain size it can be 256x 256 six pixel kind of a resolution and then each pixel is either having a certain grade of color and all that is fed into the neural network and it then gets trained in and it's able to base on these pixels pixel information it is able to get trained and able to recognize the features and extract the features and thereby it is able to identify these images and the location of these images and then when you feed a new image it kind of based on the training it will be able to figure out where this images from so that's the way a little bit under the hood how it works so what are we going to do in this tutorial we will see what is deep learning and what do we need for deep learning and one of the main components of deep learning is neural network so we 6:07:58 will see what is neural network what is a perceptron and how to implement logic gates like and or and so on using perceptrons the different types of neural networks and then applications of deep learning and we will also see how neural networks works so how do we do the training of neural networks and at the end we will end up with a small demo code which will take you through intensive flow now in order to implement deep learning code there are multiple libraries or development environments that are a available and tensor flow is one of them so the focus at the end of this would be on how to use tensor flow to write a piece of code using python as a programming language and we will take up a an example which is a very common one which is like the hollow world of deep learning the handwriting number recognition which is a mnist commonly known as mnist database so we will take 6:09:00 a look at amness database and how we can train a neural network to recognize handwritten numbers so that's what you will see in this particular video so let's get started what is deep learning deep learning is like a subset of what is known as a high level concept called artificial intelligence you must be already familiar must have heard about this term artificial intelligence so artificial intelligence is like the high level concept if you will and in order to implement artificial intelligence application s we use what is known as machine learning and within machine learning a subset of machine learning is deep learning machine learning is a little bit more generic concept and deep learning is one type of machine learning if you will and we will see a little later in maybe the following slides a little bit more in detail how deep learning is different from traditional 6:10:00 machine learning but to start with we can mention here that deep learning uses one of the differentiators between deep learning and traditional machine learning is that deep learning uses neural networks and we will talk about what are neural networks and how we can Implement neural networks and so on and so forth as a part of this tutorial so a little deeper into deep learning deep learning primarily involves working with complicated unstructured data compared to traditional machine learning with where we normally use structured data in deep learning the data would be primarily images or Voice or maybe text file so and it is large amount of data as well and deep learning can handle complex operations it involves complex operations and the other difference between traditional machine learning and deep learning is that the feature extraction happens pretty much automatically in traditional machine learning feature engineering is done manually the data scientists we data 6:11:00 scientists have to do feature engineering feature extraction but in deep learning that happens automatically and of course deep learning for large amounts of data complicated unstructured data deep learning gives very good performance now as I mentioned one of the secret sources of deep learning is neural networks let's see what neural networks is neural networks is based on our biological neurons the whole concept of deep learning and artificial intelligence is based on on human brain and human brain consists of billions of tiny stuff called neurons and this is how a biological neuron looks and this is how an artificial neuron looks so neural networks is like a simulation of our human brain human brain has billions of biological neurons and we are trying 6:12:01 to simulate the human brain using artificial neurons this is how a biological neuron looks it has tendres and the corresponding component with an artificial neural network is or an artificial neuron are the inputs they receive the inputs through ddes and then there is the cell nucleus which is basically the processing unit in a way so in artificial neuron also there is a piece which is an equivalent of this cell nucleus and based on the weights and B es we will see what exactly weights and biases are as we move the input gets processed and that results in an output in a biological neuron the output is sent through a synapse and in an artificial neuron there is an equivalent of that in the form of an output and biological neurons are also interconnected so there are billions of neurons which are interconnected in the same way artificial neurons are also interconnected so this output of this 6:13:02 neuron will be fed as an input to another the neuron and so on now in neural network one of the very basic units is a perceptron so what is a perceptron A perceptron can be considered as one of the fundamental units of neural networks it can consist at least one neuron but sometimes it can be more than one neuron but you can create a perceptron with a single neuron and it can be used to perform certain functions it can be used as a basic basic binary classifier it can be trained to do some basic binary classification and this is how a basic perceptron looks like and this is nothing but a neuron you have inputs X1 X2 X to xn and there is a summation function and then there is what is known as an activation function and based on this input what is known as the weighted sum the activation function either gets 6:14:03 gives an output like a zero or one so we say the neuron is either activated or not so that's the way it works so you get the inputs these inputs are each of the inputs are multiplied by a weight and there is a bias that gets added and that whole thing is fed to an activation function and then that results in an output and if the output is correct it is accepted if it is wrong if there is an error then that error is fed back and the neuron then adjust the weights and biases to give a new output and so on and so forth so that's what is known as the training process of a neuron or a neural network there's a concept called perceptron learning so perceptron learning is again one of the very basic learning processes the way it works is somewhat like this so you have all these inputs like X1 to xn and each of these inputs is multiplied by a weight and then that sum this is the formula or the equation so that sum w ixi 6:15:05 Sigma of that which is the sum of all these product of X and W is added up and then a bias is added to that the bias is not dependent on the input but or the input values but the bias is common for one neuron however the bias value keeps changing during the training process once the training is completed the values of these weights W1 W2 and so on and the value of the bias gets fixed so that is basically the whole training process and that is what is known as the perceptron training so the weights and biases keep changing till you get the accurate output and the summation is of course passed through the activation function as you see here this wixi summation plus b is passed through activation function and then the neuron gets either fired or not and based on that there will be an output that output is compared with the actual or expected 6:16:08 value which is also known as labeled information so this is the process of supervised learning so the output is already known and um that is compared and thereby we know if there is an error or not and if there is an error the error is fed back and the weights and biases are updated accordingly till the error is reduced to the minimum so this iterative process is known as pertron learning or perceptron learning Rule and this error needs to be minimized so till the error is minimized this iteratively the weights and biases keep changing and that is what is the training process so the whole idea is to update the weights and the bias of the perceptron till the error is minimized the error need not be zero the error may not ever reach zero but but the idea is to keep changing these weights and bi so that the error 6:17:09 is minimum the minimum possible that it can have so this whole process is an iterative process and this is the iteration continues till either the error is zero which is uh unlikely situation or it is the minimum possible Within These given conditions now in 1943 two scientists Warren mik and Walter Pitts came up with an experiment where they were able to implement the logical functions like and or and nor using neurons and that was a significant breakthrough in a sense so they were able to come up with the most common logical Gates they were able to implement some of the most common logical Gates which could take two inputs Like A and B and then give a corresponding result so for example in case of an and gate A and B and then the output is a in case of an R gate it is a 6:18:11 plus b and so on and so forth and they were able to do this using a single layer perceptron now most of these GS it was possible to use single layer perceptron except for XR and we will see why that is in a little bit so this is how an and gate works the inputs A and B the output should be fired or the neuron should be fired only when both the inputs are one so if you have 0 0 the output should be zero for 01 it is again 0 1 0 again 0 and 1 one the output should be one so how do we implement this with a neuron so it was found that by changing the values of Weights it is possible to achieve this logic so for example if we have equal weights like 7. 7 and then if we take the sum of the weighted product so for example example 7 into 0 and then 7 into 0 will give you 0 and so on and so forth and in the last 6:19:11 case when both the inputs are one you get a value which is greater than one which is the threshold so only in this case the neuron gets activated and output is there is an output in all the other cases there is no output because the threshold value is one so this is implementation of an hand gate using a single perceptron or a single neuron similarly an orgate in order to Implement an orgate in case of an orgate the output will be one if either of these inputs is one so for example 01 will result in one or other in all the cases it is one except for 0 0 so how do we implement this using a perceptron once again if you have a perceptron with weights for example 1.2 now if you see here if in the first case when both are zero the output is zero in the second case when it is 0 and 1 1.2 into 0 is 0 and then 1. 2 into 1 is 1 and in the second case similarly the output is 1.2 in the last case when both the inputs 6:20:12 are 1 the output is 2.4 so during the training process these weights will keep changing and then at one point where the weights are equal to W1 is equal 1.2 and W2 is equal to 1.2 the system learns that it gives the correct output so that is implementation of orgate using a single neuron or a single layer perceptron now Exar gate this was one of the challenging ones they tried to implement an Exar gate with a single level perceptron but it was not possible and therefore in order to implement an XR so this was like a a roadblock in the progress of U neural network however subsequently they realize that this can be implemented an XR gate can be implemented using a multi-level perceptron or MLP so in this case there are two layers instead of a single layer and this is how you can Implement an XR gate so you will see that X1 and X2 are 6:21:13 the inputs and there is a hidden layer and that's why it is denoted as H3 and H4 and then you take the output of that and feed it to the output at 05 and provide a threshold here so we will see here that this is the numerical calculation so the weights are in this case for X1 it is 20 and minus 20 and once again 20 and minus 20 so these inputs are fed into H3 and H4 so you'll see here for H3 the input is 0 1 1 1 and for H4 it is 1 1 1 and if you now look at the output final output where the threshold is taken as one if we use a sigmoid with the threshold one you will see that in these two cases it is zero and in the last two cases it is one so this is a implementation of XR in case of XR only when one of the inputs is one you will get an output so that is what 6:22:14 we are seeing here if we have either both the inputs are one or both the inputs are zero then the output should be zero so that is what is an exclusive or gate so it is exclusive because only one of the inputs should be one and then only you'll get an output of one which is Satisfied by this condition so this is a special implementation XR gate is a special implementation of perceptron now that we got a good idea about perceptron let's take a look at what is the neural network so we have seen what is a perceptron we have seen what is a neuron so we will see what exactly is a neural network so neural network is nothing but a network of these neurons and they are different types of neural networks there are about five of them these are artificial neural network convolutional neural network then recursive neural network or recurrent neural network deep neural network and deep belief Network so and each of these types of neural networks have a special you know they 6:23:14 can solve special kind of problems for example convolutional neural networks are very good at performing image processing and image recognition and so on whereas RNN are very good for speech recognition and also text analysis and so on so each type has some special characteristics and they can they're good at performing certain special kind of tasks what are some of the applications of deep learning deep learning is today used extensively in gaming you must have heard about alphao which is a game created by a startup called Deep Mind which got acquired by Google and alphago is an AI which defeated the human world champion lead all in this game of Go so gaming is an area where deep learning is being extensively used and a lot of research happens in the area of gaming as well in addition to that nowadays there are 6:24:16 neural networks or special type called generative adversarial networks which can be used for synthesizing either images or music or text and so on and they can be used to compose music so the neural network can be trained to compose a certain kind of music and autonomous cars you must be familiar with Google Google's self-driving car and today a lot of Automotive companies are investing in this space and uh deep learning is a core component of this autonomous Cars the cars are trained to recognize for example the road the the lane markings on the road signals any objects that are in front any obstruction and so on and so forth so all the involves deep learning so that's another major application and robots we have seen several robots including Sofia you may be familiar with sopia who was given a citizenship by Saudi Arabia and 6:25:19 there are several such robots which are very humanlike and the underlying technology in many of these robots is deep learning medical Diagnostics and Health Care is another major area where deep learning is being used and within Healthcare Diagnostics again there are multiple areas where deep learning and image recognition image processing can be used for example for cancer detection as you may be aware if cancer is detected early on it can be cured and one of the challenges is in the availability of Specialists who can diagnose cancer using these diagnostic images and various scans and and so on and so forth so the idea is to train neural network to perform some of these activities so that the load on the cancer specialist doctors or oncologists comes down and there is a lot of research happening here and there are already quite a few applications that 6:26:20 are claimed to be performing better than human beings in this space can be lung cancer it can be breast cancer and so on and so forth so Healthcare is a major area where deep learning is being applied let's take a look at the inner working of a neural network so how does an artificial neural network let's say identify can we train a neural network to identify the shapes like squares and circles and triangles when these images are fed so this is how it works any image is nothing but it is a digital information of the pixels so in this particular case let's say this is an image of 28x 28 pixel and this is an image of a square there's a certain way in which the pixels are lit up and so these pixels have a certain value maybe from 0 to 256 and 0 indicates that it is black or it is dark and 256 indicates it 6:27:22 is completely it is white or lit up so that is like an indication or a measure of the how the pixels are lit up and so this is an image is let's say consisting of information of 784 pixels so all the information what is inside this image can be kind of compressed into the 784 pixels the way each of these pixels is lit up provides information about what exactly is the image so we can train neural networks to use that information and identify the images so let's take a look how this works so each neuron the value if it is close to one that means it is white whereas if it is close to zero that means it is black now this is a an animation of how this whole thing works so these pixels one of the ways of doing it is we can flatten this image 6:28:25 and take this complete 784 pixels and feed that as input to our neural network the neural network can consist of probably several layers there can be a few hidden layers and then there is an input layer and an output layer now the input layer take the 784 pixels as input the values of each of these pixels and then you get an output which can be of three types or three classes one can be a square a circle or a triangle now during the training process there will be initially obviously you feed this image and it will probably say it's a circle or it will say it's a triangle so as a part of the training process we then send that error back and the weights and the biases of these neurons are adjusted till it correctly identifies that this is a square that is the whole training mechanism that happens out here now let's take a look at a circle 6:29:25 same way so you feed these 784 pixels there is a certain pattern in which the pixels are lit up and the neural network is trained to identify that pattern and during the training process once again it would probably initially identify it incorrectly saying this is a square or a triangle and then that error is fed back and the weights and biases are adjusted finally till it finally gets the image correct so that is the training process so now we will take a look at same way a triangle so now if you feed another image which is consisting of triangles so this is the training process now we have trained our neural network to classify these images into a triangle or a circle and a square so now this neural network can identify these three types of objects now if you feed another image 6:30:29 and it will be able to identify whether it's a square or a triangle or a circle now what is important to be observed is that when you feed a new image it is not necessary that the image or the the triangle is exactly in this position now the neural network actually identifies the patterns so even if the triangle is let's say positioned here not exactly in the middle but maybe at the corner or in the side it would still identify that it is a triangle and that is the whole idea behind pattern recognition so how does this training process work this is a quick view of how the training process works so we have seen that a neuron consists of inputs it receives inputs and then there is a weighted sum which is nothing but this XI wi summation of that plus the bias and this is then fed to the activation function and that in turn gives us a output now during the 6:31:31 training process initially obviously ly when you feed these images when you send maybe a square it will identify it as a triangle and when you maybe feed a triangle it will identify as a square and so on so that error information is fed back and initially these weights can be random maybe all of them have zero values and then it will slowly keep changing so the as a part of the training process the values of these weights W1 W2 up to WN keep changing in such a way that towards the end of the train process it should be able to identify these images correctly so till then the weights are adjusted and that is known as the training process so and these weights are numeric values could be 0. 52535 and so on it could be positive or it could be negative and the value that is coming here is the pixel value as we have seen it can be anything between 0 to 1 you can scale it between 0 to 1 or 0 to 256 whichever way 0 being black and 6:32:33 256 being white and then all the other colors in between so that is the input so these are numerical values this multiplication or the product W ixi is a numerical value and the bias is also a numerical value you need to keep in mind that the bias is fixed for a neuron it doesn't change with the inputs whereas the weights are one per input so that is one important point to be noted so but the bias also keeps changing initially it will again have a random value but as a part of the training process the weights the values of the weights W1 W2 WN and the value of B will change and ultimately once the training process is complete these values are fixed for this particular neuron W1 W2 up to WN and plus the value of the B is also fixed for this particular neuron and in this way there will be multiple neurons and each there may be multiple levels of neurons here and that's the way the training process works so this another 6:33:34 example of multi-layer so there are two hidden layers in between and then you have the input layer values coming from the input layer then it goes through multiple layers hidden layers and then there is an output layer and as you can see there are weights and biases for each of these neurons in each layer and all of them gets keeps changing during the training process and at the end of the training process all these weights have a certain value and that is a trained model and those values will be fixed once the training is completed all right then there is something known as activation function neural networks consists of one of the components in neural networks is activation function and every neuron has an activation function and there are different types of activation functions that are used it could be a relu it could be sigmoid and so on and so forth and the activation function is what decides whether a neuron should be fired or not so the output should be zero or one is decided 6:34:36 by the activation function and the activation function in turn takes the input which is the weighted sum remember we talked about wixi + B that weighted sum is fed as a input to the activation function and then the output can be either a zero or a one and there are different types of activation functions which are covered in an earlier video you might want to watch all right so as a part of the training process we feed the inputs the labeled data or the training data and then it gives an output which is the predicted output by the network which we indicate as y hat and then there is a labeled data because we for supervised learning we already know what should be the output so that is the actual output and in the initial process before the training is complete obviously there will be error so that is measured by what is known as a cost function so the difference between the predicted output and the actual output is the error and U the cost function can be defined in different ways there are 6:35:38 different types of cost functions so in this case it is like the average of the squares of the error so and then all the errors are added which can sometimes be called as sum of squares sum of square errors or ssse and that is then fed as a feedback in what is known as backward propagation or back propagation and that helps in the network at adjusting the weights and biases and so the weights and biases get updated till this value the error value or the cost function is minimum now there is a optimization technique which is used here called gradient descent optimization and this algorithm Works in a way that the error which is the cost function needs to be minimized so there's a lot of mathematics that goes behind this for example they find the local Minima the global Minima using the differentiation 6:36:39 and so on and so forth but the idea is this so as a training process as the as the part of training the whole idea is to bring down the error which is like let's say this is the function the cost function at certain levels it is very high the cost value of the cost function the output of the cost function is very high so the weights have to be adjusted in such a way and also the bias of course that the cost function is minimized so there is this optimization technique called gradient descent that is used and this is known as the learning rate now gradient descent you need to specify what should be the learning rate and the learning rate should be optimal because if you have a very high learning rate then the optimization will not converge because at some point it will crossover to the side on the other hand if you have very low learning rate then it might take 6:37:40 forever to convert so you need to come up with the optimum value of the learning rate and once that is done using the gradient descent optimization the error function is reduced and that's like the end of the training process all right so this is another view of gradient descent so this is how it looks this is your cost function the output of the cost function and that has to be minimized using gradient descent algorithm and these are like the parameters and weight could be one of them so initially we start with certain random values so cost will be high and then the weights keep changing and in such a way that the cost function needs to come down and at some point it may reach the minimum value and then it may increase so that is where the gradient desent algorithm decides that okay it has reached the minimum value and it will kind of try to stay here this is known as the global Minima now sometimes these curves may not be just for 6:38:42 explanation purpose this has been drawn in a nice way but sometimes these curves can be pretty erratic there can be some local Minima here and then there is a peak and then and so on so the whole idea of gradient descent optimization is to identify the global Minima and to find the weights and the bias at that particular point so that's what is gradient descent and then this is another example so you can have these multiple local minimum so as you can see at this point when it is coming down it may appear like this is a minimum value but then it is not this is actually the global minimum value and the gradient desent algorithm will make an effort to reach this level and not get stuck at this point so the algorithm is already there and it knows how to identify this global minimum and that's what it does during the training process now in order to implement deep learning there are multiple platforms and languages that are available but the most common 6:39:43 platform nowadays is tensor flow and so that's the reason we have uh this tutorial we have created this tutorial for tensorflow so we will take you through a quick demo of how to write a tensorflow code using Python and tensorflow is uh an open source platform created by Google so let's just take a look at the details of tensorflow and so this is a a library a python Library so you can use python or any other languages it's also supported in other languages like Java and R and so on but python is the most common language that is used so it is a library for developing deep learning applications especially using neural networks and it consists of primarily two parts if you will so one is the tensors and then the other is the graphs or the flow that's the way the name that's the reason for this kind of a name called tensor flow so what are tensors tensors are like 6:40:45 multi-dimensional arrays if you will that's one way of looking at it so usually you have a one-dimensional array so first of all you can have what is known as a scalar which means a number and then you have a onedimensional array something like this which means this is like a set of numbers so that is a onedimensional AR then you can have a two-dimensional array which is like a matrix and beyond that sometimes it gets difficult so this is a three dimensional array but tens oflow can handle many more Dimensions so it can have multi-dimensional arrays that is the strength of tensor flow and which makes computation deep learning computation much faster and that's the reason why tensor flow is used for developing deep learning applications so tensor flow is a deep learning tool and this is the way it works so the data basically flows in the form of tensors and the way the programming works as well is that you first create a graph of how to execute 6:41:46 it and then you actually execute that particular graph in the form of what is known as a session we will see this in the tensorflow code as we move forward so all the data is managed or manipulated in tensors and then the processing happens using these graphs there are certain terms called like for example ranks of a tensor the rank of a tensor is like a dimensional dimensionality in a way so for example if it is scalar so there is just a number just one number the rank is supposed to be zero and then it can be a onedimensional vector in which case the rank is supposed to be one and then you can have a two-dimensional Vector typically like a matrix then and in that case we say the rank is two and then if it is a three-dimensional array then rank is three and so on so it can have more than three as well so it is possible that you can store multi-dimensional arrays in 6:42:48 the form of tensors so what are some of the properties of tensor flow I think today it is one of the most popular platform tensor flow is the most popular deep learning platform or Library it is open source it's developed by Google developed and maintained by Google but it is open source one of the most important things about tensorflow is that it can run on CPUs as well as gpus GPU is a graphical Processing Unit just like CPU is central processing unit now in earlier days GPU was used for primarily for graphics and that's how the name has come and one of the reasons is that it cannot perform generic activities very efficiently like CPU but it can perform iterative actions or computations extremely fast and much faster than a CPU so they are really good for computational activities and in deep learning there is a lot of iterative computation that happens so in the form of matrix multiplication and so 6:43:50 on so gpus are very well suited for this kind of computation and tens oflow supports both GPU as well as CPU and there's a certain way of writing code in tensor flow we will see as go into the code and of course tens oflow can be used for traditional machine learning as well but then that would be an Overkill but just for understanding it may be a good idea to start writing code for a normal machine learning use case so that you get a hang of how tensorflow code works and then you can move into neural networks so that is um just a suggestion but if you're already familiar with how tensorflow works then probably yeah you can go straight into the neural networks part so in this tutorial we will take the use case of recognizing handwritten digits this is like a hollow world of deep learning and this is a nice little Ms database is a nice little database that has images of handwritten digits 6:44:53 nicely formatted because very often in deep learning and neural networks we end up spending a lot of time in preparing the data for training and with amness database we can avoid that you already have the data in the right format which can be directly used for training and mnist also offers a bunch of built-in utility functions that we can straight away use and call those functions without worrying about writing our own functions and that's one of the reasons why amness database is very popular for training purposes initially when people want to learn about deep learning and tensor flow this is the database that is used and it has a collection of 70,000 handwritten digits and a large part of them are for training then you have test just like in any machine learning process and then you have validation and all of them are labeled so you have the images and their 6:45:54 label and these images they look somewhat like this so they are handwritten images collected from a lot of individuals people have these are samples written by human beings they have handwritten these numbers these numbers going from 0 to 9 so people have written these numbers and then the images of those have been taken and formatted in such a way that it is very easy to handle so that is amness database and the way we are going to implement this in our tens oflow is we will feed this data especially the training data of along with the label information and uh the data is basically these images are stored in the form of the pixel information as we have seen in one of the previous slides all the images are nothing but these are excels so an image is nothing but an arrangement of pixels and the value of the pixel either it is lit up or it is not or in somewhere in between that's 6:46:55 how the images are stored and that is how they are fed into the neural network and for training once the network is trained when you provide a new image it will be able to identify within a certain error of course and for this we will use one of the simpler neural network configurations called softmax and for Simplicity what we will do is we will flatten these pixels so instead of taking them in a two-dimensional arrangement we just flatten them out so for example it's starts from here it is at 28x 28 so there are 7484 pixels so pixel number one starts here it goes all the way up to 28 then 29 starts here and goes up to 56 and so on and the pixel number 784 is here so we take all these pixels flatten them out and feed them like one single line into our neural network and this is a 6:47:58 what is known as a softmax layer what it does is once it is trained it will be able to identify what digit this is so there are in this output layer there are 10 neurons each signifying a digit and at any given point of time when you feed an image only one of these 10 neurons gets activated so for example if this is strained properly and if you feed a number nine like this then this particular neuron gets AC activate it so you get an output from this neuron let me just use uh a pen or a laser to show you here okay so you're feeding a number nine let's say this has been trained and now if you're feeding a number nine this will get activated now let's say you feed one to the trained Network then this neuron will get activated if you feed two this neuron will get activated 6:48:59 and so on I hope you get the idea so this is one type of a neural network or an activation function known as softmax layer so that's what we will be using here this is one of the simpler ones for quick and easy understanding so this is how the code would look we will go into our lab environment in the cloud and uh we will show you there directly but very quickly this is how the code looks and uh let me run you through briefly here and then we will go into the Jupiter notebook where the actual code is and we will run that as well so as a first step first of all we are using python here and that's why the syntax of the language is Python and the first step is to import the tensorflow library so and we do this by using this line of code saying import tensor flow as TF TF is just for convenience so you can name give any name and once you do this TF is tensor flow is available as an object in 6:49:59 the name of TF and then you can run its uh method and accesses its attributes and so on and so forth and mnus database is actually an integral part of tensorflow and that's again another reason why we as a first step we always use this example mnist database example so you just simply import mnist database as well using this line of code and you slightly modify this so that the labels are in this format what is known as one hot true which means that the label information is stored like an array and uh let me just uh use the pen to show what exactly it is so when you do this one hot true what happens is each label is stored in the form of an array of 10 digits and let's say the number is uh 8 okay so in this case all the remaining values there will be a bunch of zeros so 6:51:00 this is like array at position 0 this is at position one position two and so on and so forth let's say this is position 7 then this is position 8 that will be one because our input is eight and again position 9 will be zero okay so one hot encoding this one hot encoding true will kind of load the data in such a way that the labels are in such a way that only one of the digits has a value of one and that indicates So based on which digit is one we know what is the label so in this case the eighth position is one therefore we know this sample data the value is eight similarly if you have a two here let's say then the labeled information will be somewhat like this so you have your labels so you have this as zero the zeroth position the first position is also zero the second position is one because this indicates number two and then you have third as 6:52:01 zero and so on okay so that is the significance of this one hot true all right and then we can check how the data is uh looking by displaying the the data and as I mentioned earlier this is pretty much in the form of digital form like numbers so all these are like pixel values so you will not really see an image in this format but there is a way to visualize that image I will show you in a bit and uh this tells you how many images are there in each set so the training there are 55,000 images in training and in the test set there are 10,000 and then validation there are 5,000 so altoe there are 70,000 images all right so let's uh move on and we can view the actual image by uh using the matplot clip library and this is how you can view this is the code for viewing the images and you can view them in color or you you can view them in Gray 6:53:03 scale so the cmap is what tells in what way we want to view it and what are the maximum values and the minimum values of the pixel values so these are the Max and minimum values so of the pixel values so maximum is one because this is a scaled value so one means it is uh White and zero means it is black and in between is it can be anywhere in between black and white and the way to train the model there is a certain way in which you write your tensorflow code and um the first step is to create some placeholders and then you create a model in this case we will use the softmax model one of the simplest ones and um placeholders are primarily to get the data from outside into the neural network so this is a very common mechanism that is used and uh then of course you will have variables which are your remember these are your weights and biases so for in our case there are 10 6:54:06 neurons and each neuron actually has 784 because each neuron takes all the inputs if we go back to our slide here actually every neuron takes all the 784 inputs right this is the first neuron it has it receives all the 784 this is the second neuron this also receives all the 78 so each of these inputs needs to be multiplied with a weight and that's what we are talking about here so these are this is a a matrix of 784 values for each of the neurons and uh so it is like a 10 by 784 Matrix because there are 10 neurons and uh similarly there are biases now remember I mentioned biases only one per neuron so it is not one per input unlike the weights so therefore there are only 10 biases because there are only 10 neurons in this case so that is what we are 6:55:07 creating a variable for biases so this is something little new in tensor flow you will see unlike our regular programming languages where everything is a variable here the variables can be of three different types you have placeholders which are primarily used for feeding data you have variables which can change during the course of computation and then a third type which is not shown here are constants so these are like fixed numbers all right so in a regular programming language you may have everything as variables or at the most variables and constants but in tens oflow you have three different types placeholders variables and constants and then you create what is known as a graph so tensorflow programming consists of graphs and tensors as I mentioned earlier so this can be considered ultimately as a tensor and then the graph tells how to execute the whole implementation so that the execution is stored in the form of a graph and in this case what we are doing is we are 6:56:08 doing a multiplication TF you remember this TF was created as a tensorflow object here one more level one more so TF is available here now tensor flow has what is known as a matrix multiplication or mmal function so that is what is being used here in this case so we are using the matrix multiplication of tensor flow so that you multiply your input values x with W right this is what we were doing x w + B you just adding B and this is in very similar to one of the earlier slides where we saw Sigma XI wi so that's what we are doing here matrix multiplication is multiplying all the input values with the corresponding weights and then adding the bias so that is the graph we created and then we need to Define what is our loss function and what is our Optimizer so in this case we again use the tensor flows apis so TF do 6:57:10 NN softmax cross entropy with logits is the uh API that we will use and reduce mean is what is like the mechanism whereby which says that you reduce the error and Optimizer for doing deduction of the error what Optimizer are we using so we are using G radian descent Optimizer we discussed about this in couple of slides uh earlier and for that you need to specify the learning rate you remember we saw that there was a a slide somewhat like this and then you define what should be the learning rate how fast you need to come down that is the learning rate and this again needs to be tested and tried and to find out the optimum level of this learning rate it shouldn't be very high in which case it will not converge or shouldn't be very very low because it will in that case it will take very long so you define the optimizer and then you call the method minimize for that Optimizer 6:58:12 and that will Kickstart the training process and so far we've been creating the graph and in order to actually execute that graph we create what is known as a session and then we run that session and once the training is completed we specify how many times how many iterations we want it to run so for example in this case we are saying th000 steps so that is a exit strategy in a way so you specify the exit condition so a training will run for th000 iterations and once that is done we can then evaluate the model using some of the techniques shown here so let us get into the code quickly and see how it works so this is our Cloud environment now you can install tensorflow on your local machine as well I'm showing this demo on our existing Cloud but you can also install denslow on your local machine and uh there is a separate video on how to set up your tsor flow environment you 6:59:13 can watch that if you want to install your local environment or you can go for other any cloud service like for example Google Cloud Amazon or Cloud Labs any of these you can use and U run and try the code okay so it has got started we will log in all right so this is our deep learning tutorial uh code and uh this is our tensorflow environment and uh so let's get started the first we have seen a little bit of a code walk through uh in the slides as well now you will see the actual code in action so the first thing we need to do is import tensor flow and then we will import the data and we need to adjust the data in such a way that the one hot is encoding is set to True one hot 7:00:14 encoding right as I explained earlier so in this case the label values will be shown appropriately and if we just check what is the type of the data so you can see that this is a data sets python data sets and if we check the number of images the way it looks so this is how it looks it is an array of type float 32 similarly the number if you want to see what is the number of training images there are 55,000 then there are test images 10,000 and then validation images 5,000 now let's take a quick look at the data itself visualization so we will use um matte plot clip for this and um if we take a look at the shape now shape gives us like the dimension of the tensors or or or the arrays if you will so in this case the training data set if we see the size of the training data set using the 7:01:15 method shape it says there are 55,000 and 55,000 by 784 so remember this 784 is nothing but the 28 by 28 28 into 28 so that is equal to 784 so that's what it is uh showing now we can take just uh one image and just see what is the the first image and see what is the shape so again size obviously it is only 784 similarly you can look at the image itself the data of the first image itself so this is how which it shows so large part of it will probably be zeros because as you can imagine in the image only certain areas are written rest is U black so that's why you will mostly see zeros either it is black or white but then there are these values are so the values are actually they are scaled so the values are between 0o and one okay so this is what you're seeing so certain locations there are some values and then other locations there are zeros so that 7:02:18 is how the data is stored and loaded if we want to actually see what is the value of the handwritten image if you want to view it this is how you view it so you create like do this reshape and um matplot lib has this um feature to show you these images so we will actually use the function called um IM am show and then if you pass this parameters appropriately you will be able to see the different images now I can change the values in this position so which image we are looking at right so we can say if I want to see what is there in maybe I ,000 right so 5,000 has three similarly you can just say five what is in five five as eight what is in 50 again H so basically by the way if 7:03:19 you're wondering uh how I'm executing this code shift enter in case you're not familiar with Jupiter node books shift enter is how you execute each cell individual cell and if you want execute the entire program you can go here and say run all so that is how this code gets executed and um here again we can check what is the maximum value and what is the minimum value of this pixel values as I mentioned this is it is scaled so therefore it is between the values lie between 1 and zero now this is where we create our model the first thing is to create the required placeholders and variables and that's what we are doing here as we have seen in the slides so we create one placeholder and we create two variables which is for the weights and biases these two variables are actually matrices so each variable has 784 by 7:04:22 10al values okay so one for this 10 is for each neuron there are 10 neurons and 784 is for the pixel values inputs that are given which is 28 into 28 and the biases as I mentioned one for each neuron so there will be 10 biases they are stored in a variable by the name b and this is the graph which is basically the multiplication of these matrix multiplication of X into W and then the bias is added for each of the neurons and the whole idea is to minimize the error so let me just execute I think this code is ex executed then we Define what is our the Y value is basically the label value so this is another placeholder we had X as one placeholder and Yore true as a second placeholder and this will have values in the form of uh 10 digigit 10 digigit uh arrays and 7:05:23 uh since we said one hot encoded the position which has a one value indicates what is the label for that particular number all right then we have cross entropy which is nothing but the loss loss function and we have the optimizer we have chosen gradient descent as our Optimizer then the training process itself so the training process is nothing but to minimize the cross entropy which is again nothing but the loss function so we Define all of this in the form of a graph so this up to here remember what we have done is we have not exactly executed any tensorflow code till now we are just preparing the graph the execution plan that's how the tensorflow code works so the whole structure and format of this code will be completely different from how we normally do programming so even with 7:06:25 people with programming experience may find this a little difficult to understand it and it needs quite a bit of practice so you may want to view this uh video also maybe a couple of times to understand this flow because the way tensorflow programming is done is slightly different from the normal programming some of you who let's say have done uh maybe spark programming to some extent will be able to easily understand this um but even in spark the the programming the code itself is pretty straightforward behind the scenes the EXE execution happens slightly differently but in tens oflow even the code has to be written in a completely different way so the code doesn't get executed uh in the same way as you have written so that that's something you need to understand and a little bit of practice is needed for this so so far what we have done up to here is creating the variables and feeding the variables 7:07:26 and um or rather not feeding but setting up the variables and uh the graph that's all different finding maybe the uh what kind of a network you want to use for example we want to use softmax and so on so you have created the variables have to load the data loaded the data viewed the data and prepared everything but you have not yet executed anything in tens of flow now the next step is the execution in tens of flow so the first step for doing any execution in tensor flow is to initialize the variable so anytime you have any variables defined in your code you have to run this piece of code always so you need to basically create what is known as a a node for initializing so this is a node you still are not yet executing anything here you just created a node for the initialization so let us go ahead and create that and here onwards is where you will actually execute your code uh 7:08:27 intensive flow and in order to execute the code what you will need is a session tensor flow session so TF do session will give you a session and there are a couple of different ways in which you can do this but one of the most common methods of doing this is with what is known as a with Loop so you have a with tf. session as SS and with a uh colon here and this is like a block starting of the block and these indentations tell how far this block goes and this session is valid till this block gets executed so that is the purpose of creating this width block this is known as a width block so with tf. session as SS you say cs. run in it now cs. run will execute a node that is specified here so for 7:09:27 example here we are saying cs. run cess is basically an instance of the session right so here we are saying tf. session so an instance of the session gets created and we are calling that cess and then we run a node within that one of the nodes in the graph so one of the nodes here is in it so we say run that particular node and that is when the initialization of the variables happens now what this does is if you have any variables in your code in our case we have W is a variable and B is a variable so any variables that we created you have to run this code you have to run the initialization of these variables otherwise you will get an error okay so that is the that's what this is doing then we within this width block we specify a for Loop and we are saying we want the system to iterate for thousand steps and perform the training 7:10:32 that's what this for Loop does run training for thousand iterations and what it is doing basically is it is fetching the data or these images remember there are about 50,000 images but it cannot get all the images in one shot because it will take up a lot of memory and performance issues will be there so this is a very common way of Performing deep learning training you always do in batches so we have have maybe 50,000 images but you always do it in batches of 100 or maybe 500 depending on the size of your system and so on and so forth so in this case we are saying okay get me 100 uh images at a time and get me only the training images remember we use only the training data for training purpose and then we use test data for test purpose you must be familiar with machine learning so you must be aware of this but in case you are not in machine learning learning also not this is not specific to deep 7:11:34 learning but in machine learning in general you have what is known as training data set and test data set your available data typically you will be splitting into two parts and using the training data set for training purpose and then to see how well the model has been trained you use the test data set to check or test the validity or the accuracy of the model so that's what we are doing here and You observe here that we are actually calling an mnist function here so we are saying mnist train. nextt batch right so this is the advantage of using mes database because they have provided some very nice helper functions which are readily available otherwise this activity itself we would have had to write a piece of code to fetch this data in batches that itself is a a lengthy exercise so we can avoid all that if we are using amness database and that's why we use this for the initial learning phase okay so when we say fetch what it will do is it will 7:12:35 fetch the images into X and the labels into Y and then you use this batch of 100 images and you run the training so says. run basically what we are doing here is we are running the training mechanism which is nothing but it passes this through the neural network passes the images through the neural network finds out what is the output and and if the output obviously the initially it will be wrong so all that feedback is given back to the neural network and thereby all the W's and B's get updated till it reaches thousand iterations in this case the exit criteria is th000 but you can also specify probably accuracy rate or something like that for the as an exit criteria so here it is it just says that okay this particular image was wrongly predicted so you need to update your weights and biases that's the feedback given to each neuron and that 7:13:36 is run for thousand iterations and typically by the end of this thousand iterations the model would have learned to recognize these handwritten images obviously it will not be 100% accurate okay so once that is done after so this happens for thousand iterations once that is done you then test the accuracy of these models by using the test data set right so this is what we are trying to do here the code may appear a little complicated because if you're seeing this for the first time you need to understand uh the various methods of tensorflow and so on but it is basically comparing the output with what has been what is actually there that's all it is doing so you have your test data and uh you're trying to find out what is the actual value and what is the predicted value and seeing whether they they are equal or not TF do equal right and how many of them are correct and so on and so forth and based on that 7:14:38 the accuracy is uh calculated as well so this is the accuracy and uh that is what we are trying to see how accurate the model is in predicting these uh numbers or these digits okay so let us run this this entire thing is in one cell so we will have to just run it in one shot it may take a little while let us see and uh not bad so it has finished the thousand iterations and what we see here as an output is the accuracy so we see that the accuracy of this model is around 91% okay now which is pretty good for such a short exercise within such a short time we got 90% accuracy however in real life this is probably not sufficient so there are other ways in to increase in the accuracy we will see probably in some of the later tutorials how to improve this accuracy how to change maybe the hyperparameters like 7:15:40 number of neurons or number of layers and so on and so forth and uh so that this accuracy can be increase Beyond 90% hello and welcome to the tensorflow object detection API tutorial in this video I will walk you through the tensorflow code to perform object detection in a a video so let's get started this part is basically you're importing all the libraries we need a lot of these libraries for example lumpi we need image IO datetime and pill and so on and so forth and of course mat plot lib so we import all these libraries and then there are a bunch of variables which have some paths for the files and folders so this is regular stuff let's keep moving then we import the Mac plot lib and and make it in line and uh a few more Imports all right and then these are some warnings we can just ignore them so if I run this code once 7:16:40 again it'll go away all right and then here onwards we do the model preparation and what we're going to do is we're going to use an existing neural network model so we are not going to train a new one because that really will uh take a long time and uh it needs a lot of computation resources and so on and it is really not required already models that have been trained and in this case it is the SSD with mobile net that's the model that we are going to use and uh this model is trained to detect objects and uh it is readily available as open source so we can actually use this and if you want to use other models there are a few more models available so you can click on this link here and uh let me just take you there there are a few more models but we have chosen this particular one because the this is faster it may not be very accurate but that is one of the faster models but on this link you will see a lot of other models that are readily available these 7:17:41 are trained models some of them would take a little longer but they may be more accurate and so on so you can probably play around with these other models okay so we will be using that model so this piece of code this line is basically importing that model and this is also known as uh Frozen model the term we use is frozen model so we import download and import that and then we will actually use that model in our code all right so these two cells we have downloaded and imported the model and then once it is available locally we will then load this into our program all right so we are loading this into memory and uh you need to perform a couple of addition steps which is basically we need to to map the numbers to text as you may be aware when we actually build the model and when we run predictions 7:18:41 the model will not give a text the output of the model is usually a number so we need to map that to a text so for example if the network predicts that the output is five we know that five means it is an airplane things like that so this mapping is done in in this next cell all right so let's keep moving and then we have a helper code which will basically load the data or load the images and transform into numpy arrays this is also used in doing object detection in images so we are actually going to reuse because video is nothing but it consists of frames which in turn are images so we are going to pretty much use reuse the same code which we used for doing object detection in images so this is where the actual detection starts so here this is the path for where the images are stored so this is here once again we are reusing the code which we wrote for detecting 7:19:43 objects in an image so this is the path where the images were stored and this is the extension and this was done for about two or three images so we will continue to use this and uh we go down I'll skip this section so this is the cell where we are actually loading the video and converting it into frames and then using frame by frame we are detecting the objects in the image so in this code what we are doing basically is there a few lines of code what they do is basically once they find an object a box will be drawn around those uh each of those objects and the input file the name of the input video file is uh traffic it is the extension is MP4 and uh we have this video reader it's a excellent object which is basically part of this class called image IO so we can read and write videos using that and uh the video that we are going to use is traffic. MP4 you can use any mp4 file 7:20:43 but in our case I picked up video which has uh like car so let me just show you so this is in this object detection folder I have this mp4 file I'll just quickly prate this video it's a little slow yeah okay so here we go this is the video it's a short one relatively small video so that for this particular demo and what it will do is once we run our code it will detect each of these cars and it will annotate them as cars so in this particular video we only have cars we can later on see with another video I think I have cat here so we can also try with that but let's first check with this uh traffic video so let me go back so we will be reading this frame by frame and um know actually we will be reading the video file but then we will be analyzing it frame by frame and we will be reading them at 10 frames per second that is the rate we are mentioning here and analyzing it and then annotating and then writing it back 7:21:45 so you will see that we will have a video file named something like this traffic annotated and um we will see the annotated video so let's go back and run through this piece of code and then we will come back and see the annotated uh video this might take a little while so I will pause the video after running this particular cell and then come back to show you the results all right so let's go ahead and run it so it is running now and it is also important that at the end you close the video writer so that it is similar to a file pointer when you open a file you should also make sure you close it so that it doesn't hog the resources so it's very similar the end of it the last piece or last line of code should be video writer. close all right so I'll pause and then I'll come back okay so I will see you in a little bit all right so now as you can see here the processing is done the ourglass is disappeared that 7:22:47 means the video has been processed so let's go back and check the annotated video we go back to my file manager so this was the original traffic. MP4 and now you have here traffic uncore annotated MP4 so let's go and run this and see how it looks you see here it has got each of these cars are getting detected let me pause and show you so we pause here it says car 70% let us allow it to go a little further it detects something on top what is that truck okay so I think because of the board on top it somehow thinks there is a truck let's lay some more and see if it detects anything else so this is again a car looks like so let us yeah so this is a car and it has confidence level of 69% okay this is again a car all right so basically till the end it goes and detects each and every car that is passing by now we can 7:23:49 quickly repeat this process for another video let me just show you the other video which is a cat again there is uh this cat is not really moving or anything but it is just standing there staring and moving a little slowly and uh our application will our network will detect that this is a cat and uh even when the cat moves a little bit in the other direction it'll continue to detect and show that it is a cat Okay so yeah so this is how the original video is let's go ahead and change our code to analyze this one and see if it detects our Network detects this cat close this here we go and I'll go back to my code mode all we need to do is change this traffic to cat the extension it will automatically pick up because it is given here and then it will run through so very quickly once again what it is doing is this video reader video uncore reader has a a neat little feature or interface whereby you can say for frame 7:24:51 in video reader so it will basically provide frame by frame so you are in a loop frame by frame and then you take that each frame that is given to you you take it and analyze it as if it is an image individual image so that's the way it works so it is very easy to handle this all right so now let's once again run just this cell rest of the stuff Remains the Same so I will run this cell again it will take a little while so the our glasses come back I will pause and then come back in a little while all right so the processing is done let's go and check the annotated video go here so we have get annotated before let's play this all right so you can see here it is detecting the cat and in the beginning you also saw it detected something else here there looks like it detected one more object so let's just go back and see what it has detected here let's see yes so what is it trying to show here it's too small not able to see but uh it is trying to 7:25:53 detect something I think it is saying it is a car I don't know all right okay so in this video there's only pretty much only one object which is the cat and uh let's wait for some time and see if it continues to detect it when the cat turns around and moves as well just in a little bit that's going to happen and we will see there we go and in spite of turning the other way I think our network is able to detect that it is a cat so let me freeze and then show here it is actually still continues to detect it as a cat all right so that's pretty much it I think that's the only object that it detects in this particular video okay so close this so that's pretty much it thank you very much for watching this video and you have a great day and in case you have any questions please uh put them below the video here and we will be more than happy to get back to you and make sure you put your email ID so that we can contact you in case you have any questions thank you once again 7:26:55 bye-bye today we're going to be covering the convolutional neural network tutorial do you know how deep learning recognizes the objects in an image and really this particular neural network is how image recognition works it's very Central one of the biggest building blocks for image recognition it does it using convolution neural network and we over here we have the basic picture of a u hummingbird pixels of an image fed as input you have your input layer coming in so it takes that graphic and puts it into the input layer you have all your hidden layers and then you have your output layer and your output layer one of those is going to light up and say oh it's a bird we're going to go into depth we're going to actually go back and forth on this a number of times today so if you're not catching all the image um don't worry we're going to get into the details so we have our input layer accepts the pixels of the image as input in the form of arrays and you can see up here where they've actually um labeled each block of the bird in different arrays so we'll dive into deep as to how that looks like and how those matrixes are set up your hidden layer carry out 7:27:56 feature extraction by performing certain calculations and manipulations so this is the part that kind of reorganizes that picture multiple ways until we get some data that's easy to read for the neural network this layer uses a matrix filter and performs convolution operation to detect patterns in the image and if you remember that convolution means to coil or to twist so we're going to twist the data around and alter it and use that operation to detect the new pattern there are multiple hidden layers like convolution layer real U is how that is pronounced and that's the rectified linear unit that has to do with the activation function that's used pooling layer also uses multiple filters to detect edges corners eyes feathers beak Etc and just like the term says pooling is pulling information together and we'll look into that a lot closer here so if you're if it's a little confusing now we'll dig in deep and try to get you uh squared away with that and then finally there is a fully connected layer that identifies 7:28:57 the object in the image so we have these different layers coming through in the hidden layers and they come into the final area and that's where we have say one node or one neural network entity that lights up that says it's a bird what's in it for you we're going to cover an introduction to the CNN what is convolution neural network how CNN recognizes images we're going to dig deeper into that and really look at the individual layers in the convolutional neural network and finally we do a use case implementation using the CNN we'll begin our introduction to the CNN by introducing ja pioneer of convolutional neural network Yan leun he was the director of Facebook AI research group built the first convolutional neural network called lenette in 1988 so these have been around for a while and have had a chance to mature over the years it was used for character recognition tasks like reading zip code digits imagine processing mail and automating that process CNN is a feed forward neural network that is generally used to analyze visual images by 7:29:58 producing data with a grid like topology a CNN is also known as a convet and very key to this is we are looking at images that was what this was designed for and you'll see the different layers as we dig in Mirror some of the other some of them are actually now used since we're using uh tensor flow and carass and our code later on you'll see that some of those layers appear in a lot of your other neural network Frameworks uh but in this case this is very Central to processing images and doing so in a variety that captures multiple images and really drill down into their different features in this example here you see flowers of two varieties Orchid and a rose I think the Orchid is much more dainty and beautiful and the rose smells quite beautiful I have a couple rose bushes in my yard uh they go into the input layer that data is then sent to all the different nodes in the next layer one of the Hidden layers based on its different weights and its setup it then comes out and gives those a new value those values then are multiplied by their weights and go to the next hidden layer and so on and then you have the output layer and one of those notes 7:31:00 comes out and says it's an orchid and the other one comes out and says it's a rose depending on how was well it was trained what separates the CNN or the convolutional neural network from other neural networks is a convolutional operation forms the basis of any convolutional neural network in a CNN every image is represented in the form of arrays of pixel values so here we have a real image of the digit 8 uh that then gets put onto its pixel values represented in the form of an array this case you have a two-dimensional array and then you can see in the final in form we transform the digit 8 into its representational form of pixels of zeros and on where the ones represent in this case the black part of the eight and the zeros represent the white background to understand the convolution neural network or how that convolutional operation Works we're going to take a side step and look at matrixes in this case we're going to simplify it we're going to take two matrices A and B of one dimension now kind of separate this from your thinking as we learned that 7:32:01 you want to focus just on the Matrix aspect of this and then we'll bring that back together and see what that looks like when we put the pieces for the convolutional operation here we've set up two arrays we have uh in this case are a single Dimension Matrix and we have a = 5 37597 and we have Bal 123 so in the convolution as it comes in there it's going to look at these two and we're going to start by doing multiplying them a * B and so we multiply the the arrays element wise and we get 5 6 6 where five is the 5 * 1 6 is 3 * 2 and then the other six is 2 * 3 and since the two arrays aren't the same size they're not the same setup we're going to just truncate the first one and we're going to look at the second array multiplied just by the first three elements of the first array now that's going to be a little confusing remember a computer gets to repeat these processes hundreds of times so we're not going to just forget those other numbers later on we'll see we'll bring those back in and 7:33:03 then we have the sum of the product in this case 5 + 6 + 6 = 17 so in our a * B our very first digit in that Matrix of a * B is 17 and if you remember I said we're not going to forget the other digits so we now have 325 we move one set over and we take 325 and we multiply that times B and you'll see that 3 * 1 is 3 2 * 2 is 4 and so on and so on WE sum it up so now now we have the second digit of our a * B product in The Matrix and we continue on with that same thing so on and so on so then we would go from uh 375 to 759 to 597 this short Matrix that that we have for a we've now covered all the different entities in a that match three different levels of B now in a little bit we're going to cover where we use this math at this multiplying of matrixes and how that works uh but it's important to understand that we're going through through the Matrix and multiplying the different parts to it to match the 7:34:03 smaller Matrix with the larger Matrix I know a lot of people get lost at is you know what's going on here with these matrixes uh oh scary math not really that scary when you break it down we're looking at a section of a and we're comparing it to be so when you break that down your mind like that you realize okay so I'm I'm just taking these two matrixes and comparing them and I'm bringing the value down into one Matrix a * B we're producing that information in a way that will help the computer see different aspects let's go ahead and flip over again back to our images here we are back to our images talking about going to the most basic two-dimensional image you can get to consider the following two images the image for the symbol backslash when you press the back slash the above image is processed and you can see there for the image for the forward slash is the opposite so when we click the forward slash button that flips uh very basic we have four pixels going in can't get any more basic than that here we have a little bit more complicated picture we take a real image of a smiley face um 7:35:04 then we represent that in the form of black and white pixels so if this was an image in the computer it's black and white and like we saw before we convert this into the zeros and one so where the other one would have just been a matrix of just four dots now we have a significantly larger image coming in so don't worry we're going to bring this all together here in just a little bit layers in convolutional neural network when we're looking at this we have our convolution layer and that really is the central aspect of processing images and the convolutional neural network that's why we have it and then that's going to be feeding in and you have your reu layer which is you know as we talked about the rectified linear unit we'll talk about that a little bit layer the reu is an how it Act is how that layer is activated is the math behind it what makes the neurons fire you'll see that in a lot of other neural networks when you're using it just by itself it's for processing smaller amount ounts of data where you use the atom activation feature for large data coming in now 7:36:05 because we're processing small amounts of data in each image the reu layer works great you have your pooling layer that's where you're pulling the data together pooling is a neural network term it's very commonly used I like to use the term reduce so if you're coming from the map and reduce side you'll see that we're mapping all this data through all these networks and then we're going to reduce it we're going to pull it together and then finally we have the fully connected layer that's where our output's going to come out so we have started to look at matrixes we've started to look at the convolutional layer and where it fits in and everything we've taken a look at images so we're going to focus more on the convolution layer since this is a convolutional neural network a convolution layer has a number of filters and perform convolution operation every image is considered as a matrix of pixel values consider the following 5x5 image whose pixel values are only zero and one now obviously when we're dealing with with color there's all kinds of things that come in on color processing but we want to keep it simple and just keep it black and white 7:37:05 and so we have our image pixels uh so we're sliding the filter Matrix over the image and Computing the dot product to detect the patterns and right here you're going to ask where does this filter come from this is a bit confusing because the filter is going to be derived uh later on we build the filters when we program or train our model so you don't need to worry what the filter actually is but you do need to understand how a convolution layer works is what is the filter doing filter and you'll have many filters you don't have just one filter you'll have lots of filters that are going to look for different aspects and so the filter might be looking for just edges it might be looking for different parts we'll cover that a little bit more detail in a minute right now we're just focusing on how the filter works as a matrix remember earlier we talked about multiplying matrixes together and here we have our two-dimensional Matrix and you can see we take the filter and we multiply it in the upper left image and you can see right here 1 Time 1 1 time 0 1 Time 1 we multiply those all together 7:38:06 then sum them and we end up with a convolved feature of four we going to take that and sliding the filter Matrix over the image and Computing the dot product to detect patterns so we're just going to slide this over we're going to predict the first one and slide it over one notch predict the second one and so on and so on all the way through until we have a new Matrix and this Matrix which is the same size as a filter has red reduce the image and whatever filter whatever that's filtering out it's going to be looking at just those features reduced down to a smaller uh Matrix so once the feature maps are extracted the next step is to move them to the reu layer so the real you layer The Next Step first is going to perform an element wise operation so each of those Maps coming in if there's negative pixels so it sets all the negative pixels to zero um and you can see this nice graph to where just zeros out the negatives and then you have a value that goes from zero up to whatever value is coming out of the Matrix this introduces 7:39:07 nonlinearity to the network uh so up until now we have a we say linearity we're talking about the fact that the feature has a value so it's a linear feature this feature um came up and has let's say the feature is the edge of the beak you know it's like or the backs slash that we saw um you'll look at that and say okay this feature has a value from -10 to 10 in this case if it was one and say yeah this might be a beak it might not might be an edge right there a minus five means no we're not even going to look at it a zero and so we end up with an output and the output takes all these feature all these filtered features remember we're not just running one filter on this we're running a number of filters on this image and so we end up with an rectified feature map that is looking at just the features coming through and how they weigh in from our filters so here we have an input of a looks like a 2 can bird very exotic looking real image is scanned in multiple convolution and the reu layers for locating features and you 7:40:08 can see up here is turned it into a black and white image and in this case we're looking in the upper right hand corner for a feature and that box scans over a lot of times it doesn't scan one pixel at a time a lot of times it will Skip by two or three or four pixels uh to speed up the process that's one of the ways you can compensate if you don't have enough resources on your computation for large images and it's not just one filter slowly goes across the image uh you have multiple filters that been programmed in there so you're looking at a lot of different filters going over the different aspects of the image and just sliding across there and forming a new Matrix one more aspect to note about the reu layer is we're not just having one reu coming in uh so not only do we have multiple features going through but we're generating multiple reu layers for locating the features that's very important to note you know so we have a quite a bundle we have multiple filters multiple railu uh which brings us to the next step forward propagation now we're going to look at 7:41:08 the pooling layer the rectified feature map now goes through a pooling layer pooling is a down sampling operation that reduces the dimensionality of the feature map that's all we're trying to do we're trying to take a huge amount of information and reduce it down to a single answer this is a specific kind of bird this is an iris this is a rose so you have a rectified feature map and you see here we have a rectified feature map coming in um we set the max pooling with a 2X two filters and a stride of Two And if you remember correctly I talked about not going one pixel at a time uh well that's where the stride comes in we end up with a 2 by two pulled feature map but instead of moving one over each time and looking at every possible combination we skip a we skip a few there we go by two we skip every other pixel and we just do every other one um and this reduces our rectified feature map which as you can see over here 16x 16 to a 4x4 so we're continually trying to filter and reduce our data so that we can get to something we can manage and 7:42:10 over here you see that we have the Max uh 3 41 and two and in the max pooling we're looking for the max value a little bit different than what we were looking at before so coming from the rectified feature we're now finding the max value and then we're pulling those features together so instead of think of this as image of the map think of this as how valuable is a feature in that area how much of a feature value do we have and we just want to find the best or the maximum feature for that area they might have that one piece of the filter of the beak said oh I see a one in this beak in this image and then it skips over and says I see a three in this image and says oh this one is rated as a four we don't want to sum it together cuz then you know you might have like five ones and I'll say ah five but you might have uh four zeros and one 10 and that 10 says well this is definitely a beak where the ones will say probably not a beak a little strange analogy since we're looking at a bird but you can see how that pulled feature map comes down and we're just looking for the max value in each one of those matrixes pooling 7:43:11 layer uses different filters to identify different parts of the image like edges corners body feathers eyes beak Etc um I know I focus mainly on the beak but obviously uh each feature could be each a different part of the bird coming in so let's take a look at what that looks like structure of a convolution neural network so far this is where we're at right now we have our input image coming in and then we use our filters and there's multiple filters on there that are being developed to kind of twist and change that data and so we multiply the matrixes we take that little filter maybe it's a 2 by two we multiply it by each piece of the image and if we step two then it's every other piece of the image that generates multiple convolution layers so we have a number of convolution layers we have um setup in there just looking at that data we then take those convolution layers we run them through the reu setup and then once we've done through the ru set up and we have multiple reu going on multiple layers that are relu then we're going to take those multiple layers and 7:44:12 we're going to be pooling them so now we have the pooling layers or multiple poolings going on up until this point we're dealing with u sometimes it's multiple Dimensions you can have three dimensions some strange data setups that aren't doing images but looking at other things they can have four five six seven dimensions uh so right now we're looking at 2D image Dimensions coming in into the pooling layer so the next step is we want to reduce those Dimensions or flatten them so flattening flattening is a process of converting all of the resultant two-dimensional arrays from pulled feature map into a single long continuous linear Vector so over here you see where we have a pulled feature map maybe that's the Birdwing and it has values 6847 and we want to just flatten this out and turn it into 6847 or a single linear vector and we find out that not only do we do each of the pulled feature Maps we do all of them into one long linear Vector so now we've gone through our convolutional neural network part and we have the input layer into the next setup all we've done is 7:45:14 taken all those different pooling layers and we flatten them out and combine them into a single linear Vector going in so after we've done the flattening we have just a quick recap because we've covered so much so it's important to go back and take a look at each of the steps we've gone through the of the network so far is we have our convolution where we twist it and we filter it and multiply the matrixes we end up with our convolutional layer which uses the reu to figure out the values going out into the pooling and you have numerous convolution layers that then create numerous pooling layers pulling that data together which is the max value which one we want to send forward we want to send the best value and then we're going to take all of that from each of the pooling layers and we're going to flatten it and we're going to combine them into a single IM input going into the final layer once you get to that step you might be looking at that going boy that looks like the normal input to most neural network and you're correct it is so once we have the flattened Matrix from the pooling layer that becomes our input so the pulling 7:46:15 layer is fed as an input to the fully connected layer to classify the image and so you can see as our flattened Matrix comes in in this case we have the pixels from the flatten Matrix fed as an input back to our twocan or whatever that kind of bird that is um I need one of these to identify what kind of bird that is it comes into our forward propagation network uh and that will then have the different weights coming down across and then finally it selects that that's a bird and that it's not a dog or a cat in this case even though it's not labeled the final layer there in red is our output layer our final output layer that says bird cat or dog so quick recap of everything we've covered so far we have our input image which is twisted and multiply the filters are multiplied times the uh matri the two matrixes multiplied all the filters to create our convolution layer our convolution layers there's multiple layers in there because it's all building multiple layers off the different filters then goes through the reu as this activation and that creates our pooling and so once we get into the 7:47:16 pooling layer we then on the pooling look for who's the best what's the max value coming in from our convolution and then we take that layer and we flatten it and then it goes into a fully connected layer or fully connected neural Network and then to the output and here we can see the entire process how the CNN recognizes a bird this is kind of nice cuz it's showing the little pixels and where they're going you can see the filter is generating this convolution network and that filter shows up in the bottom part of the convolution network and then based on that it uses the relo for the pooling the pooling then find out which one's the best and so on all the way to the fully connected layer at the end or the classification in the output layer so that'd be a classification neural network at the end so we covered a lot of theory up until now and you can imagine each one of these steps has to be broken down in code so putting that together can be a little complicated not that each step of the process is overly complicated but because we have so many steps uh we have 1 two 3 four five 7:48:16 different steps going on here with substeps in there we're going to break that down and walk through that in code so in our use case implementation using the CNN we'll be using the CFR 10 data set for from Canadian Institute for advanced research for classifying images across 10 categories Unfortunately they don't let me know whether there's going to be a tan or some other kind of bird but we do get to find out whether it can categorize between a ship a frog deer bird airplane automobile cat dog horse truck so that's a lot of fun and if you're looking anything in the news at all of our automated cars and everything else you can see where this kind of processing is so important in today's world and very Cutting Edge as as far as what's coming out in the commercial deployment I mean this is really cool stuff we're starting to see this just about everywhere in Industry uh so great time to be playing with this and figuring it all out let's go ahead and dive into the code and see what that looks like when we're actually writing our script before we go on let's do uh 7:49:17 one more quick look at what we have here let's just take a look at data batch one keys and remember in Jupiter notebook I can get by with not doing the print statement if I put a variable down there it'll just display the variable and you can see under data batch one for the keys since this is a dictionary we have the batch one label data and file names uh so you can actually see how it's broken up in our data set so for the next step or step four as we're calling it uh we want to display the image using matplot Library there's many ways to display the images you could even uh well there's other ways to drill into it but matplot library is really good for this and we'll also look at our first reshape uh setup or shaping the data so you can have a little glimpse into what that means uh so we're going to start by importing our map plot and of course since I am doing Jupiter notebook I need to do the map plot inline command so it shows up on my page so here we go we're going to import matplot library. pyplot as PLT and if you remember map plot Library the P plot is like a canvas that 7:50:19 we paint stuff onto and there's my percentage sign M plot library in line so it's going to show up in my notebook and then of course we're going to import numpy as NP for our numbers python array setup and let's go ahead and set um x equals to data batch one so this will pull in all the data going into the x value and then because this is just a long stream of binary data uh we need to go a little bit of reshaping so in here we have to go ahead and reshape the data we have 10,000 images okay that looks correct and this is kind of an interesting thing it took me a little bit to I had to go research this myself to figure out what's going on with this data and what it is is it's a 32x 32 picture and let me do this let me go ahead and do a drawing pad on here uh so we have 32 bits by 32 bits and it's in color so there's three bits of color now I don't know why the data is particularly like this it probably has to do with how they originally encoded it but most pictures put the three 7:51:21 afterward so what we're doing here is we're going to take uh the shape we're going to take the data which is just a long stream of information and we're going to break it up into 10,000 pieces and those 10,000 pieces then are broken into three pieces each and those three pieces then are 32x 32 you could look at this like an oldfashioned projector where they have the red screen or the red projector the blue projector and the green projector and they add them all together and each one of those is a 32x 32 bit so that's probably how this was originally formatted with in that kind of Ideal things have changed so we're going to transpose it and we're going to take the three which was here and we're going to put it at the end so the first part is reshaping the data from a single line of bit data or whatever format it is into 10,000x 3x 32x 32 and then we're going to transpose the color factor to the last place so it's the image then the 32x 32 in the middle that's this part right here and then finally we're going to take this uh which is three 7:52:23 bits of data and put it at the end so it's more like we do we process images now and then as type this is really important that we're going to use an integer 8 you can come in here and you'll see a lot of these they'll try to do this with a float or a float 64 what you got to remember though is a float uses a lot of memory so once you switch this into uh something that's not integer 8 which is goes up to 128 you are just going to the the amount of ram let just put that in here is going to go way up the amount of ram that it loads uh so you want to go ahead and use this you can try the other ones and see what happens if you have a lot of RAM on your computer but for this exercise this will work just fine and let's go ahead and take that and run this so now our X variable is all loaded and it has all the images in it from the batch one data batch one and just to show we were talking about with the as type on there if we go ahead and take x0 and just look for its max value let me go ahead and 7:53:25 run that uh you'll see it doesn't oops I said 128 it's 255 uh you'll see it doesn't go over 255 because it's an basically an asky character is what we're keeping that down to we're keeping those values down so they're only 255 0 to 255 versus uh float value which would bring this up um exponentially in size and since we're using the map plot Library we can do um oops that's not what I wanted since we're using the map plot Library we can take our canvas and just do a PLT do IM for image show and uh let's just take a look at what x0 looks like and it comes in I'm not sure what that is but you can see it's a very low grade image uh broken down to the minimal pixels on there and if we did the same thing oh let's do uh let's see what one looks like hopefully it's a little easier to see run on there not enter let's hit the run on that uh and we can see this is probably a semi that's a good guess on there and I can just go back up here instead of typing the same line in over and over and we'll look at three uh that looks like a dump 7:54:26 truck unloading uh and so on you can do any of the 10,000 images we can just jump to 55 uh looks like some kind of animal looking at us there probably a dog and just for fun let's do just one more uh uh run on there and we can see a nice car for image number four uh so you can see we pce through all the different images it's very easy to look at them and they've been reshaped to fit our view and what the uh matap plot Library uses for its format so the next step is we're going to start creating some helper functions we'll start by a one hot encoder to help us when we're processing the data remember that your labels they can't just be words they have to switch it and we use the one hot encoder to do that and then we'll also create a uh class uh CFR helper so it's going to have a knit and a setup for the images and then finally we'll go ahead and run that code so you can see what that looks like and then we get into the fun part where we're actually going to start creating our model our actual neural network model so let's start by creating our one hot encoder we're going 7:55:28 to create our own here uh and it's going to return and out and we'll have our Vector coming in and our values equal 10 what this means is that we have the 10 values the 10 possible labels and remember we don't look at the labels as a number because a car isn't one more than a horse that'd be just kind of bizarre to have horse equals zero car equals 1 plane equals 2 cat equals 3 so a cat plus a card equals what uh so instead we create a numpy array of zeros and there's going to be 10 values so we have a 10 different values in there so you have uh zero or one one means it's a cat zero means it's not a cat um in the next line it might be that uh one means it's a car zero means it's not a car so instead of having one output with a value of 0 to 10 you have 10 outputs with the values of 0 to one that's what the one hot encoder is doing here and we're going to utilize this in code in just a minute so let's go ahead and take a look at the next helpers we have a few of these helper functions we're going to 7:56:29 build and when you're working with a very complicated python project dividing it up into separate definitions and classes is very important otherwise it just becomes really ungainly to work with so let's go ahead and put in our next helper uh which is a class and this is a lot in this class so we we'll break it down here let's just start uh oops we can put a space right in there there we go that this a little bit more readable at a second space so we're going to create our class the cipher Helper and we'll start by initializing it now there's a lot going on in here so let's start with the uh nit part uh self. I equals z that'll come in and a little bit we'll come back to that in the lower part we want to initialize our training batches so when we went through this there was like a meta batch we don't need the meta batch but we do need the data batch 1 2 3 4 five and we do not want the testing batch in here this is just the self all train batches so we're going to come make an array of of all those different images and then of 7:57:29 course we left the test batch out so we have our self. test batch uh we're going to initialize the training images and the training labels and also the test images and the test labels so these are just this is just to initialize these variables in here then we create another definition down here and this is going to set up the images let's just take a look and see what's going on in there now we could have all just put this as part of the uh init part uh since this is all just helper stuff but breaking it up again makes it easier to read it also makes it easier when we start executing the different pieces to see what's going on so that way we have a nice print statement to say hey we're now running this and this is what's going on in here we're going to set up these self-training images at this point and that's going to go to a numpy array vstack and in there we're going to load up uh in this case the data for D and S all trained batches again that points right up to here so we're going to go through each one of these uh files or each one of these data sets CU they're not a file anymore we've brought them in 7:58:30 data batch one points to the actual data and so our self-training images is going to stack them all into our into a numpy array and then it's always nice to uh get the training length and that's just a total number of uh self-training images in there and then we're going to take the self trining images and let me switch marker colors because I am getting a little too much on the markers up here oops there we go bring down our marker change so we can see it a little better and it's this point this should look familiar where did we see this well we wanted to uh uh look at this above and we wanted to look at the images in the matplot library we had to reshape it so we're doing the same thing here we're taking our self-training images and uh based on the training length total number of images because we stacked them all together so now it's just one large file of images we're going to take and look at it as our our three video cameras that are each displaying uh 32x 32 we're going to switch switch that around so that now we have um each of 7:59:30 our images that stays the same place and then we have our 32x 32 and then by our three our last our three different values for the color and of course we want to go ahead and uh they run this where you say divide by 255 that was from earlier it just brings all the data into 0 to one that's what this is doing so we're turning this into a 0 to one array which is uh all the pictures 30 32x 32x 3 and then we're going to take the self-training labels and we're going to pump those through our one hot incoder we just made and we're going to stack them together and uh again we're converting this into an array that goes from uh instead of having horse equals one dog equals two and then horse plus dog would equal three which would be cat no it's going to be uh you know an array of 10 where each one is 0o to one then we want to go ahead and set up our test images and labels and when we're doing this you're going to see it's the same thing we just did with the rest of let 8:00:31 me just change colors right here this is no different than what we were doing up here with our training Set uh we're going to stack the different uh images uh we're going to get the length of them so we know how many images are in there uh you certainly could add them by hand but it's nice to let the computer do it especially if it ever changes on the other end and you're using other data and again we reshape them and transpose them and we also do the one hot encoder same thing we just did on our training images so now our test images are in the same format so now we have a definition which sets up all our images in there and then the next step is to go ahead and batch them or next batch and let's do another breakout here for batches because this is really important to understand tends to throw me for a little Loop when I'm working with tensor flow or carass or a lot of these we have our data coming in if you remember we had like 10,000 photos let me just put 10,000 down here we don't want to run all 10,000 at once so we want to break this up into batch sizes and you also 8:01:31 remember that we had the number of photos in this case uh length of test or whatever number is in there uh we also have 32 by 32 by 3 so when we're looking at the batch size we want to change this from 10,000 to um a batch of in this case I think we're going to do batches of 100 so we want to look at just 100 the first 100 of the photos and if you remember we set selfi equal to to zero uh so what we're looking at here is we're going to create X we're going to get the next batch from the very initialized we've already initialized it for zero so we're going to look at X from zero to batch size which we set to 100 so just the first 100 images and then we're going to reshape that into uh and this is important to let the data know that we're looking at 100x 32 by 32x 3 now we've already formatted it to the 32x 32x 3 this just sets everything up correctly so that X has the data in there in the correct order and the correct shape and then the Y just like 8:02:33 the X uh is our labels so our training labels again they go from zero to batch size in this case they do selfi plus batch size because the selfi is going to keep changing and then finally we increment the selfi because we have zero so we so the next time we call it we're going to get the next batch size and so basically we have X and Y X being the photograph data coming in and Y being the lay and that of course is labeled through one hot encoder so if you remember correctly if it was say horse is equal to zero it would be um uh one for the zero position since this is the horse and then everything else would be zero in here me just put lines through there there we go there's our array hard to see that array so let's go ahead and take that and uh we're going to finish loading it since this is our class and now we're armed with all this um uh our setup over here let's go ahead and load that up so we're going to create a variable CH with the cfar helper in it and then we're going to do ch. setup 8:03:34 images uh now we could have just put all the setup images under the init but by breaking this up into two parts it makes it much more readable and um also if you're doing other work there's reasons to do that as far as the setup let's go ahead and run that and you can see where it says uh setting up training images and labels setting up test images and that's one of the reasons we broke it up is so that if you're testing this out you can actually have print statements in there telling you what's going on which is really nice uh they did a good job with this setup I like the way that it was broken up in the back and then one quick note you want to remember that batch to set up the next batch say we have to run uh batch equals CH next batch of 100 because we're going to use the 100 size uh but we'll come back to that we're going to use that just remember that that's part of our code we're going to be using in a minute from the definition we just made so now we're ready to create our model first thing we want to do is we want to import our tensor flow as TF I'll just go ahead and run that so it's loaded up and you can see we got a a warning here uh that's 8:04:32 because they're making some changes it's always growing and they're going to be depreciating one of the uh values from float 64 to float type or it's treated as an NP float 64 uh nothing to really worry about cuz this doesn't even affect what we're working on because we've set all of our stuff to a 255 value or 0 to one and do keep in mind that 0 to one value that we converted to 255 is still a float value uh but it'll easily work with either the uh numpy float 64 or the numpy dtype float it doesn't matter which one it goes through so the depreciation would not affect our code as we have it and in our tensor flow uh we'll go ahead let me just increase the size in there just a moment so you can get better view of the um what we're typing in uh we're going to set a couple placeholders here and so we have we're going to set x equals TF placeholder TF float 32 we just talked about the float 64 versus the numpy float we're actually just going to keep this at at float 32 more than a significant number of decimals for what we're working with and 8:05:33 since it's a placeholder we're going to set the shape equal to and we've set it equal to none because at this point we're just hold holding the place on there we'll be setting up as we run the batches that's what the first value is and then 32x 32x 3 that's what we reshaped our data to fit in and then we have our y true equals placeholder T of float 32 and the shape equals none comma 10 10 is the 10 different lay we have so it's an array of 10 and then let's create one more placeholder we'll call this hold prob or hold probability and we're going to use this we don't have to have a shape or anything for this this placeholder is for what we call Dropout if you remember from our Theory before we drop out so many nodes that's looking at or the different values going through which helps decrease bias so we need to go ahead and put a a placeholder for that also and we'll run this so it's all loaded up in there so we have our three different place holders and since we're in tensor flow when you use carass it does some of this automatically but we're in tensor flow direct carass sits 8:06:34 on tensor flow we're going to go ahead and create some more helper functions we're going to create something to help us initialize the weights initialize our bias if you remember that each uh layer has to have a bias going in we're going to go ahead and work on our our conversional 2D our Max pool so we have our pooling layer our convolutional layer and then our normal full layer so we're going to go ahead and put those all into definitions and let's let's see what that looks like in code and you can also grab some of these helper functions from the MN the uh nist setup let me just put that in there if you're under the tensor flow so a lot of these are already in there but we're going to go ahead and do our own and we're going to create our uh a knit weights and one of the reasons we're doing this is so that you can actually start thinking about what's going on in the back end so even though there's ways to do this with an automation sometimes these have to be tweaked and you have to put in your own setup in here uh now we're not going to be doing that we're just going to recreate them for our code and let's take a look at this we have our weights 8:07:30 and so what comes in is going to be the shape and what comes out is going to be uh random numbers so we're going to go ahead and just knit some random numbers based on the shape with a standard deviation of 0.1 kind of a fun way to do that and then the TF variable uh init random distribution so we're just creating a random distribution on there that's all that is for the weights now you might change that you might have a a higher standard deviation in some cases you actually load preset weights that's pretty rare usually you're testing that against another model or something like that and you want to see how those weights configure with each other uh now remember we have our bias so we need to go ahead and initialize the bias with a constant in this case we're using 0.1 a lot of times the bias is just put in as one and then you have your weights to add on to that uh but we're going to set this as 0.1 uh so we want to return a convolutional 2d in this case a neural network this is uh would be a layer on here what's going on with the con 2D is we're taking our data coming in uh we're going to filter it strides if you 8:08:33 remember correctly strides came from here's our image and then we only look at this picture here and then maybe we have a stride of one so we look at this picture here and we continue to look at the different filters going on there the other thing this does is that we have our data coming in as 32 by 32 by 3 and we want to change this so that it's just this is three dimensions and it's going to reformat this as just two Dimensions so it's going to take this number here and combine it with the 32x 32 so this is a very important layer here CU it's reducing our data down using different means and it connects down I'm just going to jump down one here uh it goes with the convolutional layer so you have your your kind of your pre- formatting and the setup and then you have your actual convolution layer that goes through on there and you can see here we have a knit weights by the shape a knit by s shape of three because we have the three different uh here's our three again and then we return the tfnn reu 8:09:33 with the convention 2D so this convolutional uh has this feeding into it right there it's using that as part of it and of course the input is the XY plus b the bias so that's quite a mouthful but these two are the are the keys here to creating the convolutional layers there the convolutional 2D coming in and then the convolutional layer which then steps through and creates all those filters we saw then of course we have we have our pooling uh so after each time we run it through the convectional layer we want to pull the data uh if you remember correctly on the on the pool side and let me just get rid of all my marks it's getting a little crazy there and in fact let's go ahead and jump back to that slide let's just take a look at that slide over here uh so we have our image coming in we create our convolutional layer with all the filters remember the filters go um you know the filter is coming in here and it looks at these four boxes and then if it's a step let's say step two it then goes to these four boxes and then then next step and so on uh so we have our convolutional layer that we generate or convolutional layers they use the uh reu 8:10:35 function um there's other functions out there for this though the reu is the uh most the one that works the best at least so far I'm sure that will change then we have our pooling now if you remember correctly the pooling was Max uh so if we had the filter coming in and they did the multiplication on there and we have a one and maybe a two here and another one here and a three here three is the max and so out of all of these you then create an array that would be three and if the max is over here two or whatever it is that's what goes into the pooling of what's going on in our pooling uh so again we're reducing that data down we're reducing it down as small as we can and then finally we're going to flatten it out into a single array and that goes into our fully connected layer and you can see that here in the code right here we're going to create our normal full layer um so at some point we're going to take from our pooling layer this will go into some kind of flattening process and then that will be fed into the full the different 8:11:35 layers going in down here um and so we have our input size you'll see our input layer get shape which is just going to get the shape for whatever is coming in uh and then input size initial weights is also based on uh the input layer coming in and the input size down here is based on the input layer shape so we're just going to already use the shape and already have our size coming in and of course of course uh you have to make sure youit the bias always put your bias on there and we'll do that based on the size so this will return tf. matmo input layer w+b this is just a normal full layer that's what this means right down here that's what we're going to return so that was a lot of steps we went through let's go ahead and run that so those are all loaded in there and let's go ahead and uh create the layers let's see what that looks like now that we've done all the heavy lifting and everything uh we get to do all the easy part let's go ahead and create our layers we'll create a convolution layer one and two two different convolutional layers and then we'll take that we'll 8:12:35 flatten that out create a a reshape pooling in there for our reshape and then we'll have our full uh layer at the end so let's start by creating our first uh convolutional layer then we come in here and let me just run that real quick and I want you to notice on here the three and the 32 this is important because coming into this convolutional layer we we have three different channels and 32 pixels each uh so that has to be in there the four and four you can play with this is your filter size so if you remember you have a filter and you have your image and the filter slowly steps over and filters out this image depending on what your step is for this particular setup 44 is just fine that should work pretty good for what we're doing and for the size of the image and then of course at the end once you have your convolutional layer set up you also need to pull it and you'll see that the pooling is automatically set up so that it would see the different shape based on what's coming in so here we 8:13:35 have Max two 2 by two and we put in the convolutional one that we just created the convolutional layer we just created goes right back into it and that right up here as you can see is the X it's coming in from here so it knows to look at the first model and set the the data accordingly set that up so it matches and we went ahead and ran this already I think I ran let me go and run it again and if we're going to do one layer let's go ahead and do a second layer down here and it's we'll call it convo 2 it's also convolutional layer on this and you'll see that we're feeding convolutional one in the pooling so it goes from convolutional one into convolutional one pooling from convolutional one pooling into convolutional two and then from convolutional two into convolutional two pooling and we'll go ahead and take this and run this so these variables are all loaded into memory and for our flattened layer uh let's go ahead and we'll do uh since we have 64 coming out of here and we have a 4x4 going in let's do 8X 8 by 8:14:37 64 so let's do 4,096 this is going to be the flat layer so that's how many bits are coming through on the flat layer and we'll reshape this so we'll reshape our uh convo 2 pooling and that will feed into here the convo two pulling and then we're going to set it up as a single layer that's 4,096 in size that's what that means there we'll go ahead and run this so we've now created this variable the convo two flat and then we have our first full layer this is the final uh neural network where the flat layer going in and we're going to again use the uh reu for our uh setup on there on a neural network for evaluation and you'll notice that we're going to create our first full layer our normal full layer that's our definition so we created that that's creating the normal full layer and our input for the data comes right here from from the this goes right into it uh the convo to flat so this tells it how big the data is and we're going to have it come out it's going to have uh 1024 that's how big the layer is coming out we'll go ahead and 8:15:39 run this so now we have our full layer one and with the full layer one we want to also Define the full one Dropout to go with that so our full layer one comes in uh keep probability equals whole probability remember we created that earlier and the full layer one is what's coming into it and this is going backwards and training the data we're not training every weight we're only training a percentage of them each time which helps get rid of the bias so let me go ahead and run that and uh finally we'll go ahead and create a y predict which is going to equal the normal full one Dropout and 10 cu we have 10 labels in there now in this neural network we could have added additional layers that would be another option to play with you can also play with instead of 1024 you can use other numbers for the way that sets up and what's coming out and going into the next one we're only going to do just the one layer and the one layer Dropout and you can see if we did another layer it'd be really easy just to feed in the full one Dropout into full layer two and then full Layer Two Dropout would have full Layer Two feed 8:16:41 into it and then you'd switch that here for the Y prediction for right now this is great this particular data set is tried and true and we know that this will work on it and if we just type in y predict and we run that uh we'll see that this is a tensor object uh shape question mark 10 dtype 32 a quick way to double check what we're working on so now we've got all of our uh we've done a setup all the way to the Y predict which we just did uh we want to go ahead and apply the loss function and make sure that's set up in there uh create the optimizer and then uh train our Optimizer and create a variable to initialize all the global TF variables so before we dive into the um loss function let me point out one quick thing or just kind of a rehap over a couple things and that is when we're playing with this these setups um we pointed out up here we can change the 44 and use different numbers there they change your outcome so depending on what numbers you use here will have a huge 8:17:42 impact on how well your model fits and that's the same here with the 1024 also this is also another number that if you continue to raise that number you'll get um possibly a better fit you might overfit and if you lower that number you'll use less resources and generally you want to use this in um the exponential growth an exponential being two 4 8 16 and in this case the next one down would be 512 you can use any number there but those would be the ideal numbers uh when you look at this data so the next step in all this is we need to also create uh a way of tracking how good our model is and we're going to call this a loss function and so we're going to create a cross entropy loss function and so before we discuss exactly what that is let's take a look and see what we're feeding it uh we're going to feed it our labels and we have our true labels and our prediction labels uh so coming in here is where the two different uh variables we're sending in or the two different probability distributions is one that we know is 8:18:44 true and what we think it's going to be now this function right here when they talk about cross entropy uh in information Theory the cross entropy between two probability distributions over the same underlying set of events events measures the average number of bits needed to identify an event drawn from the set that's a mouthful uh really we're just looking at the amount of error in here how many of these are correct and how many of these um are incorrect so how much of it matches and we're going to look at that we're just going to look at the average that's what the mean the reduced to the mean means here so we're looking at the average error on this and so the next step is we're going to take the error we want to know our cross entropy or our loss function how much loss we have that's going to be part of how we train the model so when you know what the loss is and we're training it you feed that back into the back propagation setup and so we want to go ahead and optimize that here's our Optimizer we're going to create the optimizer using an atom Optimizer remember there's a lot of different ways of optimizing the data 8:19:44 atom's the most popular used uh so our Optimizer is going to equal the TF train atom Optimizer if you don't remember what the learning rate is let me just pop this back into here here's our learning rate when you have your weights you have all your weights and your different nodes that are coming out here's our node coming out um and it has all its weights and then the error is being prop sent back through in reverse on our neural network so we take this error and we adjust these weights based on the different formulas in this case the atom formulas what we're using we don't want to just adjust them completely we don't want to change this weight so it exactly fits the data coming through because if we made that kind of adjustment it's going to be biased to whatever the last data we sent through is in instead we're going to multiply that by 0.001 and make a very small shift in this weight so our Delta W is only 0.001 of the actual Delta W of the full change we're going to compute from the atom and then we want to go ahead and train it so our training or set up a training uh uh variable or 8:20:45 function and this is going to equal our Optimizer minimize cross entropy and we make sure we go ahead and run this so it's loaded in there and then we're almost ready to train our model but before we do that we need to create one more um variable in here and we're going to create a variable to initialize all the global TF variables and when we look at this um the TF Global variable initializer this is a tensor flow um object it goes through there and it looks at all our different setup that we have going under our tensor flow and then initializes those variables uh so it's kind of like a magic one because it's all hidden in the back end of tensor flow all you need to know about this is that you have to have the initialization on there which is an operation um and you have to run that once you have your setup going so we'll go ahead and run this piece of code and then we're going to go ahead and train our data so let me run this so it's loaded up there and so now we're going to go ahead and run the model by creating a graph session graph session is a tensorflow term so you'll see that 8:21:46 coming up it's one of the things that throws me because I always think of graphic and Spark and graph as just general graphing uh but they talk about a graph session so we're going to go ah aad and run the model and let's go ahead and walk through this uh what's going on here and let's paste this data in here and here we go so we're going to start off with the with the TF session as sess so that's our actual TF session we've created uh so we're right here with the TF uh session our session we're creating we're going to run TF Global variable initializer so right off the bat we're initializing our variables here uh and then we have for I in range 500 so what's going on here remember 500 we're going to break the date up and we're going to batch it in at 500 points each we've created our session run so we're going to do with TF session as session right here we've created our variable session uh and then we're going to run and we're going to go ahead and initialize it so we have our TF Global variables initializer that we created um 8:22:47 that initializes our our session in here the next thing we're going to do is we're going to go for I in range of 500 batch equals ch. next batch so if you remember correctly this is loading up um 100 pictures at a time and uh this is going to Loop through that 500 times so we are literally doing uh what is that uh 500 time 100 is uh 50,000 so that's 50,000 pictures we're going to process right there in the first process is we're going to do a session run we're going to take our train we created our train variable or Optimizer in there we're going to feed it the dictionary uh we had our feed dictionary that we created and we have xal ALS batch 0o coming in y true batch one hold the probability 0.5 and then just so that we can keep track of what's going on we're going to every uh 100 steps we're going to run a print So currently onstep format C accuracy is um and we're going to look at matches equals tf. equal TF 8:23:48 argument y prediction one tf. AR Max y true comma one so we're going to look at this as how many matches it has and here our ACC uh all we're doing here is we're going to take the matches how many matches they have it creates it generates a chart we're going to convert that to float that's what the TF cast does and then we just want to know the average we just want to know the average of the um accuracy and then we'll go ahead and print that out uh print session run accuracy feed dictionary so it takes all this and it prints out our accuracy on there so let's go ahead and take this oops FP screens there let's go ahead and take this and let's run it and this is going to take a little bit to run uh so let's see what happens on my old laptop and we'll see here that we have our current uh we're currently on Step Zero it takes a little bit to get through the accuracy and this will take just a moment to run we can see that on our Step Zero it has an accuracy of1 or 0.128 um and as it's running we'll go ahead you don't need to watch it run all the way but uh this accuracy is going to 8:24:49 change a little bit up and down so we've actually lost some accuracy during our step two um but we'll see how that comes out let's come come back after we run it all the way through and see how the different steps come out I was actually reading that backwards uh the way this works is the closer we get to one the more accuracy we have uh so you can see here we've gone from a 0.1 to a 39 um and we'll go ahead and pause this and come back and see what happens when we're done with the full run all right now that we've uh prepared the meal got it in the oven and pull out my finished dish here if you've ever watched uh any of the old cooking shows let's discuss a little bit about this accuracy going on here and how do you interpret that we've done a couple things first we've defined accuracy um the reason I got it backwards before is you have uh loss or accuracy and with loss you'll get a graph that looks like this it goes oops that's an S by the way there we go you get a graph that curves down like this and with accuracy you get a graph that 8:25:50 curves up this is how good it's doing now in this case uh one is supposed to be really good accuracy that means it gets close to one but it never crosses one so if you have an accuracy of one that is phenomenal um in fact that's pretty much un you know unheard of and the same thing with loss if you have a loss of zero that's also unheard of the zero is actually on this this axis right here as we go in there so how do we interpret that because you know if I was looking at this and I go oh 0. 51 that's uh 51% you're doing 50/50 no this is not percentage let me just put that in there it is not percentage uh this is l log rithmic what that means is that 0. 2 is twice as good as 0.1 and uh when we see4 that's twice as good as 0. 2 real way to convert this into a percentage you really can't say this is is a direct percentage conversion what you can do though is in your head if we were to give this a percentage uh we might look 8:26:50 at this as uh 50% we're just guessing equals 0.1 and at 50% roughly equals 0.1 that's where we started up here at the top remember at the top here here's our 0.128 the accuracy at 50% then 75% is about 0.2 and so on and so on don't quote those numbers because it doesn't work that way they say that if you have .95 that's pretty much saying 100% And if you have uh anywhere between you'd have to go look this up let me go and remove all my drawings there uh so the magic number is 0.5 we really want to be over a 0. five in this whole thing and we have uh both 0.504 remember this is accuracy if we were looking at loss then we'd be looking the other way but 0.0 you know instead of how high it is we want how low it is uh but when accuracy being over a 0.5 is pretty valid that means this is pretty solid and if you get to a 095 then it's a direct correlation that's what we're looking 8:27:51 for here in these numbers you can see we finished with this model at 051 135 so still good um and if we look at uh when they ran this on the other end remember there's a lot of Randomness that goes into it when we see the weights uh they got 05251 so a little better than ours but that's fine you'll find your own uh comes up a little bit better or worse depending on uh just that Randomness and so we've gone through the whole model we've created we trained the model and we've also gone through on every 100th run to test the model to see how accurate it is welcome to the RNN tutorial that's the the recurrent neural network so we talk about a feed forward neural network in a feed forward neural network information flows only in the forward direction from the input nodes through the hidden layers if any and to the output nodes there are no Cycles or Loops in the network and so you can see here we have our input layer I was talking about how it just goes straight forward into the hidden layers so each one of those connects and then connects to the next hidden layer it connects to the output layer and of course we have a nice simplified version where it has a 8:28:52 predicted output and they refer to the input as x a lot of times and the out put as why decisions are based on current input no memory about the past no future scope why recurrent neural network issues in feed forward neural network so one of the biggest issues is because it doesn't have a scope of memory or time a feed forward neural network doesn't know how to handle sequential data uh it only considers only the current input so if you have a series of things and because three points back affects what's happening now and what your output affects what's happening that's very important so whatever I put as an output is going to affect the next one um a feed forward doesn't look at any of that it just looks at this is what's coming in and it cannot memorize previous inputs so it doesn't have that list of inputs coming in solution to feed forward neural network you'll see here where it says recurrent neural network and we have our X on the bottom going to H going to Y that's your feed forward uh but right in the middle it has a value C so there a whole another process was memorizing what's going on in the hidden layers and 8:29:53 the hidden layers as they produce data lead into the next one so your hidden layer might have an output that goes off to Y uh but that output goes back into the next prediction coming in what this does is this allows it to handle sequential data it considers the current input and also the previously received inputs and if we're going to look at General drawings and um Solutions we should also look at applications of the RNN image captioning RNN is used to caption an image by analyzing the activities present in it a dog catching a ball in midair uh that's very tough I mean you know we have a lot of stuff that analyzes images of a dog and the image of a ball but it's able to add one more feature in there that's actually catching the ball in midair time series prediction any time series problem like predicting the prices of stocks in a particular month can be solved using RNN and we'll dive into that in our use case and actually take a look at some stock one of the things you should know about analyzing stock today is that it is very 8:30:53 difficult and if you're analyzing the whole stock stock the stock market at the New York Stock Exchange in the US produces somewhere in the neighborhood if you count all the individual trades and fluctuations by the second um it's like three terabytes a day of data so we're only to look at one stock just analyzing One stock is really tricky in here we'll give you a little jump on that so that's exciting but don't expect to get rich off of it immediately another application of the RNN is natural language processing text Mining and sentiment analysis can be carried out using RNN for natural language processing and you can see right here the term natural language processing when you stream those three words together is very different than I if I said processing language natural leave so the time series is very important when we're analyzing sentiments it can change the whole value of a sentence just by switching the words around or if you're just counting the words you might get one sentiment where if you actually look at the order they're in you get a completely different sentiment when it rains look for rainbows when it's dark look for stars both of these are 8:31:54 positive sentiments and they're based upon the order of which the sentence is going in machine translation given an input in one language RNN can be used to translate the input into a different languages as output I myself very linguistically challenged but if you study languages and your good with languages you know right away that if you're speaking English you would say big cat and if you're speaking Spanish you would say cat big so that translation is really important to get the right order to get uh there's all kinds of parts of speech that are important to know by the order of the words here this person is speaking in English and getting translated and you can see here a person is speaking in English in this little diagram I guess that's denoted by the flags I have a flag I own it no um but they're speaking in English and it's getting translated into Chinese Italian French German and Spanish languages some of the tools coming out are just so cool so somebody like myself who's very linguistically challenged I can now travel into world I 8:32:56 would never think of because I can have something translate my English back and forth readily and I'm not stuck with a communication gap so let's dive into what is a recurrent neural network recurrent neural network works on the principle of saving the output of a layer and feeding this back to the input in order to predict the output of the layer sounds a little confusing when we start breaking it down it'll make more sense and usually we have a propagation forward neural network with the input layers the hidden layers the output layer with the recurrent neural network we turn that on its side so here it is and now our X comes up from the bottom into the hidden layers into Y and they usually draw very simplified X to H with c as a loop a to Y where a B and C are the perimeters a lot of times you'll see this kind of drawing in here digging closer and closer into the H and how it works going from left to right you'll see that the C goes in and then the X goes in so the x is going Upward Bound and C is going to the right a is going out and and C is also going out that's 8:33:57 where it gets a little confusing so here we have xn uh CN and then we have y out and C out and C is based on HT minus one so our value is based on the Y and the H value are connected to each other they're not necessarily the same value because H can be its own thing and usually we draw this or we represent it as a function h of T equals a function of C where H of T minus one that's the last H output and X of T going in so it's the last output of H combined with the new input of x uh where HT is the new state FC is a function with the parameter C that's a common way of denoting it uh HT minus one is the Old State coming out and then xit T is an input Vector at time of Step T well we need to cover types of recurrent neural networks and so the first one is the most common one which is a one to one single output one: one neural network is usually known as a vanilla neural Network used for regular machine 8:34:57 learning problems why because vanilla is usually considered kind of a just a real basic flavor but because it's a very basic a lot of times they'll call it the vanilla neural network uh which is not the common term but it is you know like kind of a slang term people will know what you're talking about usually if you say that then we run one to mini so you have a single input and you might have a multiple outputs in this case uh image captioning as we looked at earlier where we have not just looking at it as a dog but a dog catching a ball in the air there and then you have many to One Network takes in a sequence of inputs examples sentiment analysis where a given sentence can be classified as expressing positive or negative sentiments and we looked at that as we were discussing if it rains look for a rainbow so positive sentiment where rain might be a negative sentiment if you were just adding up the words in there and then of course if you're going to do a one: one many to one one to many there's many to many networks takes in a sequence of inputs and generates a sequence of outputs examp example machine translation so we have a lengthy 8:35:58 sentence coming in in English and then going out in all the different languages uh you know just a wonderful tool very complicated set of computations you know if you're a translator you realize just how difficult it is to translate into different languages one of the biggest things you need to understand when we're working with this neural network is what's called The Vanishing gradient problem while training an RNN your slope can be either too small or very large and this makes training difficult when the slope is too small the problem is known as Vanishing gradient and you'll see here they have a nice U image loss of information through time so if you're pushing not enough information forward that information is lost and then when you go to train it you start losing the third word in the sentence or something like that or it doesn't quite follow the full logic of what you're working on exploding gradient problem Oh this is one that runs into everybody when you're working with this particular neural network when the slope tends to grow exponentially instead of decaying this problem is called exploding gradient 8:36:58 issues in gradient problem long tring time poor performance bad accuracy and I'll add one more in there uh your computer if you're on a a lower-end computer testing out a model will lock up and give you the memory error explaining gradient problem consider the following two examples to understand what should be the next word in the sequence the person who took my bike and blank a thief the students who got into engineering with blank from Asia and you can see in here we have our x value going in we have the previous value going forward and then you back propagate the error like you do with any neural network and as we're looking for that missing word maybe we'll have the person took my bike and blank was a thief and the student who got into engineering with a blank were from Asia consider the following example the person who took the bike so we'll go back to the person who took the bike was blank a thief in order to understand what would be the next word in the sequence the RNN must memorize the previous context whether the subject was 8:37:58 singular noun or a plural noun so was a thief is singular the student who got into engineering well in order to understand what would be the next word in the sequence the RNN must memorize the previous context whether the subject was singular noun or a plural noun and so you can see here the students who got into engineering with blank were from Asia it might be sometimes difficult for the eror to back propagate to the beginning of the sequence to predict what should be the output so when you run into the gradient problem we need a solution the solution to the gradient problem first we're going to look at exploding gradient where we have three different solutions depending on what's going on one is identity initialization so the first thing we want to do is see if we can find a way to minimize the identities coming in instead of having it identify everything just the important information we're looking at next is to truncate the back propagation so instead of having uh whatever information it's sending to the next series we can truncate what it's sending we can lower that particular uh set of 8:39:01 layers make those smaller and finally is a gradient clipping so when we're training it we can clip what that gradient looks like and narrow the training model that we're using when you have a Vanishing gradient the OPA problem uh we can take a look at weight initialization very similar to the identity but we're going to add more weights in there so it can identify different aspects of what's coming in better choosing the right activation function that's huge so we might be activating based on one thing and we need to limit that we haven't talked too much about activation functions so we look at that just minimally uh there's a lot of choices out there and then finally there's long short-term memory networks the lstms and we can make adjustments to that so just like we can clip the gradient as it comes out we can also um expand on that we can increase the memory Network the size of it so it handles more information and one of the most common problem problems in today's uh setup is what they call longterm dependencies suppose we try to predict the last word in the text the clouds are 8:40:03 in the and you probably said sky here we do not need any further context it's pretty clear that the last word is going to be Sky suppose we try to predict the last word in the text I have been staying in Spain for the last 10 years I can speak fluent maybe you said Portuguese or French no you probably said Spanish the word we predict will depend on the previous few words in context here we need the context of Spain to predict the last word in the text it's possible that the gap between the relevant information and the point where it is needed to become very larg lstms help us solve this problem so the lstms are a special kind of recurrent neural network capable of learning long-term dependencies remembering information for long periods of time is their default Behavior All recurrent neural networks have the form of a chain of repeating modules of neural network connections in standard rnns this repeating module will have a very simple structure such as a single tangent H layer lstm s's also have a chain-like 8:41:06 structure but the repeating module has a different structure instead of having a single neural network layer there are four interacting layers communicating in a very special way lstms are a special kind of recurrent neural network capable of learning long-term dependencies remembering information for long periods of time is their default Behavior LST tms's also have a chain-like structure but the repeating module has a different structure instead of having a single neural network layer there are four interacting layers communicating in a very special way as you can see the deeper we dig into this the more complicated the graphs kit in here I want you to note that you have x a t minus one coming in you have x a t coming in and you have x a t + one and you have H of T minus one and H of T coming in and H of t + 1 going out and of course on the other side is the output a um in the middle we have our tangent H but it occurs in two different places so not only when we're Computing the x of t + one are we getting the 8:42:07 tangent H from X of T but we're also getting that value coming in from the X of T minus one so the short of it is as you look at these layers not only does it does the propagate through the first layer goes into the second layer back into itself but it's also going into the third layer so now we're kind of stacking those up and this can get very complicated as you grow that in size it also grows in memory too and in the amount of resources it takes uh but it's a very powerful tool to help us address the problem of complicated long sequential information coming in like we were just looking at in the sentence and when we're looking at our long shortterm memory network uh there's three steps of processing processing in the lstms that we look at the first one is we want to forget irrelevant parts of the previous state you know a lot of times like you know is as in a unless we're trying to look at whether it's a plural noun or not they don't really play a huge part in the language so we want to get rid of them then selectively update cell State values so we only want to update the 8:43:08 cell State values that reflect what we're working on and finally we want to put only output certain parts of the cell state so whatever is coming out we want to limit what's going out too and let's dig a little deeper into this let's just see what this really looks like uh so step one decide how much of the past it should remember first step in the lstm is to decide which information to be omitted in from the cell in that particular time step it is decided by the sigmoid function it looks at the previous state h of T minus one and the current input x a t and computes the function so you can see over here we have a function of T equals the sigmoid function of the weight of f the H at T minus one and then X at t plus of course you have a bias in there within neural network so we have a bias function so f equals forget gate decides which information to delete that is not important from the previous time step considering an L STM is fed with the following inputs from the previous and present time step Alice is good in 8:44:10 physics John on the other hand is good in chemistry so previous output John plays football well he told me yesterday over the phone that he had served as a captain of his college football team that's our current input so as we look at this the first step is the forget gate realizes there might be a change in context after en counting the First full stop Compares with the current input sentence of xit so we're looking at that full stop and then Compares it with the input of the new sentence the next sentence talks about John so the information on Alice is deleted okay that's important to know so we have this input coming in and if we're going to continue on with John then that's going to be the primary information we're looking at the position of the subject is vacated and is assigned to John and so in this one we've seen that we've weeded out a whole bunch of information and we're only passing information on John since that's now the new topic so step two is in to decide how much should this unit add to the current state in the second layer there are two parts one is a sigmoid function and the other is 8:45:11 the tangent H in the sigmoid function it decides which values to let through zero or one tangent H function gives the weightage to the values which are passed is setting their level of importance minus one to one and you can see the two formulated that come up uh the I of T equals the sigmoid of the weight of I a to T minus one X of t plus the bias of I and the C of T equals the tangent of H of the weight of C of H of T minus one X of t plus the bias of C so our I of T equals the input gate determines which information to let through based on its significance in the current time step if this seems a little complicated don't worry because a lot of the programming is already done when we get to the case study understanding though that this this as part of the program is important when you're trying to figure out these what to set your settings at you should also note when you're looking at this it should have some semblance to your forward propagation neural networks where we have a value assigned to a weight plus a bias very important steps 8:46:11 than any of the neural network layers whether we're propagating into them the information from one to the next or we're just doing a straightforward neural network propagation let's take a quick look at this what it looks like from the human standpoint um as I step out in my suit again consider the current input at xft John plays football well he told me yesterday over the phone that he had served as a captain of his college football team that's our input input gate analysis the important information John plays football and he was a captain of his college team is important he told me over the phone yesterday is less important hence it is forgotten this process of adding some new information can be done via the input gate now this example is as a human form and we'll look training this stuff in just a minute uh but as a human being if I wanted to get this information from a conversation maybe it's a Google Voice listening in on you or something like that um how do we weed out the information that he was talking to me on the phone yesterday well I don't want to memorize that he talked to me on the phone yesterday or maybe that 8:47:12 is important but in this case it's not I want to know that he was the captain of the football team I want to know that he sered I want to know that John plays football and he was the captain of the college football team those are the two things that I want to take away as a human being again we measure a lot of this from the human Viewpoint and that's also how we try to train them so we can understand these neural networks finally we get to step three decides what part of the current cell State makes it to the output the third step is to decide what will be our output first we run a sigmoid layer which decides what parts of the cell State make it to the output then we put the cell State through the tangent H to push the values to be between minus1 and 1 and multiply it by the output of the sigmoid gate so when we talk about the output of T we set that equal to the sigmoid of the weight of zero of the H of T minus one you back One Step in Time by the x of t plus of course the bias the H of T equals the out of T times the tangent of the tangent h of c a t so our o equals the 8:48:13 output gate allows the past in information to impact the output in the current time step let's consider the example to predicting the next word in the sentence John played tremendously well against the opponent and one for his team for his contributions Brave blank was awarded player of the match there could be a lot of choices for the empty space current input Brave is an adjective adjectives describe a noun John could be the best output after Brave thumbs up for John awarded player of the match and if you were to pull just the nouns out of the sentence team doesn't look right because that's not really the subject we're talking about contributions you know Brave contributions or Brave team Brave player Brave match um so you look at this and you can start to train this these this neural network so it starts looking at and goes oh no Jon is what we're talking about so brave is an adjective Jon's going to be the best output and we give JN a big thumbs up and then of course we jump into my favorite part the case 8:49:15 study use case implementation of lstm let's predict the prices of stocks using the lstm network based on the stock price data between 20 2012 2016 we're going to try to predict the stock prices of 2017 and this will be a narrow set of data we're not going to do the whole stock market it turns out that the New York Stock Exchange generates roughly three terabytes of data per day that's all the different trades up and down of all the different stocks going on and each individual one uh second to second or nanc to nanc uh but we're going to limit that to just some very basic fundamental information so don't think you're going to get rich off this today but at least you can give an a you can give a step forward in how to start processing something like stock prices a very valid use for machine learning in today's markets use case implementation of lstm let's dive in we're going to import our libraries we're going to import the training set and uh get the scaling going um now if you watch any of our 8:50:16 other tutorials a lot of these pieces just start to look very familiar CU it's very similar setup uh but let's take a look at that and um just remind we're going to be using Anaconda the Jupiter notebook so here I have my anaconda Navigator when we go under environments I've actually set up a carass python 36 I'm in Python 36 and uh nice thing about Anaconda especially the newer version remember a year ago messing with Anaconda different versions of python and different environments um Anaconda now has a nice interface um and I have this installed both on a Ubuntu Linux machine and on windows so it works fine on there you can go in here and open a terminal window and then in here once you're in the terminal window this is where you're going to start uh installing using pip to install your different modules and everything now we've already pre-installed them so we don't need to do that in here uh but if you don't have them installed on your particular environment you'll need to do that and of course you don't need to use the anaconda or the Jupiter you can use whatever favorite python ID you like I'm 8:51:18 just a big fan of this because it keeps all my stuff separate you can see on this machine I have specifically installed one for carass since we're going to be working with carass under tensorflow when we go back to home I've gone up here to application and that's the environment I've loaded on here and then we'll click on the launch Jupiter notebook now I've already in my Jupiter notebook um have set up a lot of stuff so that we're ready to go kind of like uh Martha Stewarts in the old cooking shows we want to make sure we have all our tools for you so you're not waiting for them to load and uh if we go up here to where it says new you can see where you can um create a new Python 3 that's what we did here underneath the setup so it already has all the modules installed on it and I'm actually renamed this so if you go under file you can rename it we I'm calling it RNN stock and let's just take a look at start diving into the code let's get into the exciting part now we've looked at the tool and of course you might be using a different tool which is fine uh let's start putting that code in there and seeing what those Imports and uploading everything looks like now first half is 8:52:17 kind of boring when we hit the rum button because we're going to be importing numpy as NP that's uh uh the number python which is your numpy array and the matap plot library because we're going to do some plotting at the end and our pandas for our data set our pandas is PD and when I hit run uh it really doesn't do anything except for load those modules just a quick note let me just do a quick uh draw here oops shift alt there we go you'll notice when we're doing this setup if I was to divide this up oops I'm going to actually um let overlap these here we go uh this first part that we're going to do is our data prep a lot of prepping involved um in fact depending on what your system is since we're using carass I put an overlap here uh but you'll find that almost maybe even half of the code we do is all about the data prep and the reason I overlap this with uh carass let 8:53:19 me just put that down because that's what we're working in uh is because carass has like their own preset stuff so it's already pre-built in which is really nice so there's a couple Steps A lot of times that are in the Kass setup uh we'll take a look at that to see what comes up in our code as we go through and look at stock and then the last part is to evaluate and if you're working with um shareholders or uh you know classroom whatever it is you're working with uh the evaluate is the next biggest piece um so the actual code here crossed is a little bit more but when you're working with uh some of the other packages you might have like three l lines that might be it all your stuff is in your pre-processing in your data since carass has is is Cutting Edge and you load the individual layers you'll see that there's a few more lines here and cross is a little bit more robust and then you spend a lot of times uh like I said with the evaluate you want to have something you present to everybody else to say hey this is what I did this is what it looks like so let's go through those steps this is like a kind of just general overview and let's just take a look and see what the next 8:54:20 set of code looks like and in here we have a data set train and it's going to be read using the PD or pandas read CSV and it's the Google stock pric train.csv and so under this we have training set equals data set train. iocation and we've kind of sorted out part of that so what's going on here let's just take a look at let's let's look at the actual file and see what's going on there now if we look at this uh ignore all the extra files on this um I already have a train and a test set where it's sorted out this is important to notice no because a lot of times we do that as part of the pre-processing of the data we take 20% of the data out so we can test it and then we train the rest of it that's what we use to create our neural network that way we can find out how good it is uh but let's go ahead and just take a look and see what that looks like as far as the file itself and I went ahead and just opened this up in a basic word pad text editor just so we can take a look at it certainly you can open up in Excel or any other kind of spreadsheet um and we note that this is 8:55:22 a comma separated variable we have a date uh open high low close volume this is the standard stuff that we import into our stock one of the most basic set of information you can look at in stock it's all free to download um in this case we downloaded it from uh Google that's why we call it the Google stock price um and it specifically is Google this is the Google stock values from uh as you can see here we started off at 13 20102 so when we look at this first setup up here uh we have a data set train equals pdor CSV and if you noticed on the original frame um let me just go back there they had it set to home Ubuntu downloads Google stock price train I went ahead and change that because we're in the same file where I'm running the code so I've saved this particular python code and I don't need to go through any special paths or have the full path on there and then of course we want to take out um certain values in here and you're going to notice that we're using um our data set 8:56:25 and we're now in pandas uh so pandas basically it looks like a spreadsheet um and in this case we're going to do I location which is going to get specific locations the first value is going to show us that we're pulling all the rows in the data and the second one is we're only going to look at columns one and two and if you remember here from our data as we switch back on over columns we always start with zero which is the date and we're going to be looking at at open and high which would be one and two I'll just label that right there so you can see now when you go back and do this you certainly can extrapolate and do this on all the columns um but for the example let's just limit a little bit here so that we can focus on just some key aspects of stock and then we'll go up here and run the code and uh again I said the first half is very boring whenever you hit the Run button it doesn't do anything because we're still just loading the data and setting it up now that we've 8:57:27 loaded our data we want to go ahead and scale it we want to do what they call feature scaling and in here we're going to pull it up from the SK learn or the SK kit pre-processing import min max scaler and when you look at this you got to remember that um biases in our data we want to get rid of that so if you have something that's like a really high value um let's just draw a quick graph and I have something here like the maybe the stock has a value One stock has a value of 100 and another stock has a value of five um you start to get a bias between different stocks and so when we do this we go ahead and say okay 100 is going to be the Max and five is going to be the men and then everything else goes and then we change this so we just squish it down I like the word squish so it's between one and zero so 100 equals 1 or 1 equal 100 and 0 = 5 and you can just multiply it's usually just a simple 8:58:28 multiplication we're using uh multiplication so it's going to be uh minus5 and then 100 divided or 95 divided by 1 so or whatever value is is divided by 95 and uh once we've actually created our scale we've toing it's going to be from 0er to one we want to take our training set and we're going to create a training set scaled and we're going to use our scaler SC and we're going to fit we're going to fit and transform the training Set uh so we can now use the SC this this particular object we'll use it later on our testing set because remember we have to also scale that when we go to test our uh model and see how it works and we'll go ahead and click on the run again uh it's not going to have any output yet because we're just setting up all the variables okay so we pasted the data in here and we're going to create the data structure with a 60 time steps and output first note we're running 60 time steps and that is where this value here also comes in so the first thing we do is we create our X train and Y train 8:59:30 variables and we set them to an empty python array very important to remember what kind of array we're in and what we're working with and then we're going to come in here we're going to go for I in range 60 to 1258 there's our 60 60 time steps and the reason we want to do this is as we're adding the data in there there's nothing below the 60 so if we're going to use 60 time steps uh we have to start at 60 cuz it includes everything underneath of it otherwise you'll get a pointer error and then we're going to take our X train and we're going to append training set scaled this is a scaled value between zero and one and then as I is equal to 60 this value is going to be um 60 - 60 is 0 so this actually is 0 to I so it's going to be 0 to 60 1 to 61 let me just circle this part right here 1 to 61 uh 2 to 62 and so on and so on and if you remember I said 0 to 60 that's incorrect because it does not count remember it starts at zero so this is a count of 60 9:00:32 so it's actually 59 important to remember that as we're looking at this and then the second part of this that we're looking at so if you remember correctly here we go we go from uh 0 to 59 of I and then we have a comma a zero right here and so finally we're just going to look at the open value now I know we did put it in there for one to two um if you remember correctly it doesn't count the second one so it's just the open value we're looking at just open um and then finally we have y train. append training set I to zero and if you remember correctly I to or I comma zero if you remember correctly this is 0 to 59 so there's 60 values in it uh so when we do I down here this is number 60 so we're going to do this is we're creating an array and we have 0 to 59 and over here we have number 60 which is going into the Y train it's being appended on there and then this just goes all the way up so this is down here 9:01:34 is uh 0 to 59 and we'll call it 60 since that's the value over here and it goes all the way up to 1258 that's where this value here comes in that's the length of the data we're loading so we've loaded two arrays we've loaded one array that has uh which is filled with arrays from 0 to 50 9 and we loaded one array which is just the value and what we're looking at you want to think about this as a Time sequence uh here's my open open open open open open what's the next one in the series so we're looking at the Google stock and each time it opens we want to know what the next one 0o through 59 what's 60 1 through 60 what's 61 2 through 62 what's 62 and so on and so on going up and then once we've loaded those in our for Loop we go ahead and take X train and Y train equals np. array XT tr. NP array ytrain we're just converting this back into a numpy array that way we can use all the cool tools that we get with numpy array including reshaping so if we take a look 9:02:36 and see what's going on here we're going to take our X train we're going to reshape it wow what the heck does reshape mean uh that means we have an array if you remember correctly um so many numbers by 60 that's how wide it is and so we're when you when you do XT train. shape that gets one of the shapes and you get um XT train. shape of one gets the other shape and we're just making sure the data is formatted correctly and so you use this to pull the fact that it's 60 by um in this case where's that value 60 by 1199 1258 minus 60199 and we're making sure that that is shaped correctly so the data is grouped into uh 1199 by 60 different arrays and then the one on the end just means at the end because this when you're dealing with shapes and numpy they look at this as layers and so the in layer needs to be one value that's like the leaf of a tree 9:03:37 where this is the branch and then it branches out some more um and then you get the Leaf np. reshape comes from and using the existing shapes to form it we'll go ahead and run this piece of code again there's no real output and then we'll import our different Cor cross modules that we need so from Cross models we're going to import the sequential model we're dealing with sequential data we have our dense layers we have actually three layers we're going to bring in our dents our lstm which is what we're focusing on and our Dropout and we'll discuss these three layers more in just a moment but you do need the with the lstm you do need the Dropout and then the final layer will be the dents but let's go ahead and run this and that'll bring portter modules and you'll see we get an error on here and if you read it closer it's not actually an error it's a warning what does this warning mean these things come up all the time when you're working with such Cutting Edge modules that are completely being updated all the time we're not going to worry too much about the warning all it's saying is that the h5py module which is part of carass is 9:04:38 going to be updated at some point and uh if you're running new stuff on carass and you start updating your carass system you better make sure that your H5 Pi is updated too otherwise you're going to have an error later on and you can actually just run an update on the H5 Pi now if you wanted to not a big deal we're not going to worry about that today and I said we were going to jump in and start looking at what those layers mean I meant that and uh we're going to start off with initializing the RNN and then we'll start adding those layers in and you'll see that we have the lstm and then the Dropout lstm then Dropout lstm then Dropout what the heck is that doing so let's explore that we'll start by initializing the RNN regressor equals sequential because we're using the sequential model and we'll we'll run that and load that up and then we're going to start adding our lstm layer and some Dropout regularization and right there should be the Q Dropout regularization and if we go back here and remember our exploding gradient well that's what we're talking about the uh Dropout drops out 9:05:41 unnecessary data so we're not just shifting huge amounts of data through um the network so and so we go in here let's just go ahead and add this in I'll go ahead and run this and we had three of them so let me go and put all three of them in and then we can go back over them there's the second one and let's put one more in let's put that in and we'll go and put two more in I meant to put I said one more in but it's actually two more in and then let's add one more after that and as you can see each time I run these they don't actually have an output so let's take a closer look and see what's going on here so we're going to add our first lstm layer in here we're going to have units 50 the units is the positive integer and it's the dimensionality of the output space this is what's going out into the next layer here so we might have 60 coming in but we have 50 going out we have a return sequence because it is a sequence data so we want to keep that true and then you have to tell it what shape it's in well we already know the shape by just going in here and looking at xtrain shape so input shape equals the X train shape of one comma one makes it really 9:06:42 easy you don't have to remember all the numbers that put in 60 or whatever else is in there you just let it tell the regressor what model to use and so we follow our STM with a Dropout layer now understanding the Dropout layer is kind of exciting CU one of the things that happens is we can overtrain our Network that means that our neural network will memorize such specific data that it has trouble predicting anything that's not in that specific realm to fix for that each time we run through the training mode we're going to take 02 or 20% of our neurons and just turn them off so we're only going to train on the other ones and it's going to be random that way each time we pass through this we don't overtrain these nodes come back in in the next training cycle we randomly pick a different 20 and finally they see a big difference as we go from the first to the second and third and fourth the first thing is we don't have to input the shape because the shape's already the output units is 50 here this Auto The Next Step automatically knows this 9:07:42 layer is putting out 50 and because it's the next layer it automatically sets that and says oh 50 is coming out from our last layer is coming out you know goes into the regressor and of course we have our Dropout and that's what's coming into this one and so on and so on and so the next three layers we don't have to let it know what the shape is it automatically understands that and we're going to keep the units the same we're still going to do 50 units it's still a sequence coming through 50 units and a sequence now the next piece of code is what brings it all together let's go ahead and take a look at that and we come in here we put the output layer the dense layer and if you remember up here we had the three layers we had uh lstm Dropout and d uh D just says we're going to bring this all down into one output instead of putting out a sequence we just know want to know the answer at this point and let's go ahead and run that and so in here you notice all we're doing is setting things up one step at a time so far we've brought in our uh way up here we brought in our data we brought in our different modules we formatted the data for training it we' 9:08:42 set it up you know we have our y x train and our y train we have our source of data and the answers we're we know so far that we're going to put in there we' reshaped that we've come in and built our Ross we've imported our different layers and we have in here if you look we have what uh five total layers now carass is a little different than a lot of other systems because a lot of other systems put this all in one line and do it automatic but they don't give you the options of how those layers interface and they don't give you the options of how the data comes in carass is Cutting Edge for this reason so even though there's a lot of extra steps in building the model this has a huge impact on the output and what we can do with this these new models from karat so we brought in our dents we have our full model put together our regressor so we need to go ahead and compile it and then we're going to go ahead and fit the data we're going to compile the pieces so they all come together and then we're going to run our training data on there and actually recreate our regressor so it's ready to be used so let's go ahead and compile that and I can go and run 9:09:44 that and uh if you've been looking at any of our other tutorials on neural networks you'll see we're going to use the optimizer atom atom is optimized for Big Data there's a couple other optimizers out there uh beyond the scope of this tutorial but certainly Adam will work pretty good for this and loss equals mean squared value so when we're training it this is what we want to base the loss on how bad is our error well we're going to use the mean squared value for our error and the atom Optimizer for its differential equations you don't have to know the math behind them but certainly it helps to know what they're doing and where they fit into the bigger models and then finally we're going to do our fit fitting the RN into the training set we have the regressor do fit X TR TR y train epics and batch size so we know where this is this is our data coming in for the X train our y train is the answer we're looking for of our data our sequential input epic is how many times we're going to go over the whole data set we created a whole data set of X train so this is each each of those rows which includes a Time sequence of 60 and badge size another 9:10:47 one of those things where carass really shines is if you were pulling this save from a large file instead of trying to load it all into RAM it can now pick smaller batches up and load those indirectly we're not worried about pulling them off a file today this isn't big enough to uh cause the computer too much of a problem to run not too straining on the resources but as we run this you can imagine what would happen if I was doing a lot more than just one column in one set of stock in this case Google stock imagine if I was doing this across all the stocks and I had instead of just the open I had open close high low and you can actually find yourself with about 13 different variables times 60 cuz it's a Time sequence suddenly you find yourself with a gig of memory you're loading into your RAM which will just completely you know if it's just if you're not on multiple computers or cluster you're going to start running into resource problems but for this we don't have to worry about that so let's go ahead and run this and this will actually take a little bit on my computer it's an older laptop and give it a second to kick in there there we go 9:11:47 all right so we have epic so this is going to tell me it's running the first run through all the data and as it's going through it's batching them in 32 pieces so 32 lines each time and there's 1198 I think I said 1199 earlier but it's $ 1198 I was off by one and each one of these is 13 seconds so you can imagine this is roughly 20 to 30 minutes runtime on this computer like I said it's an older laptop running at uh 0.9 GHz on a dual processor and that's fine what we'll do is I'll go ahead and stop go get a drink a coffee and come back and let's see what happens at the end and where this takes us and like any good cooking show I've kind of gotten my latte I also had some other stuff running in the background so you'll see these numbers jumped up to like 19 seconds 15 seconds which you can scroll through you can see we've run it through 100 steps or 100 epics so the question is what does all this mean one of the first things you'll notice is that our loss can is over here it kind of stopped at 0.0014 you can see it kind of goes down until we hit about 0.14 three times in a row so we guessed our epic pretty 9:12:48 close since our loss has remain the same on there so to find out what we're looking at we're going to go ahead and load up our test data the test data that we didn't process yet and real stock price data set test iocation this is the same thing we did when we prepped the data in the first place so let's go ahead and go through this code and we can see we've labeled it part three making the predictions and visualizing the results so the first thing we need to do is go ahead and read the data in from our test CSV and you see I've changed the path on it for my computer and uh then we'll call it the real stock price and again we're doing just the one column here and the values from I location so it's all the rows and just the values from these that one location that's the open Stock open let's go ahead and run that so that's loaded in there and then let's go ahead and uh create we have our inputs we're going to create inputs here and this should all look familiar this is the same thing we did before we're going to take our data set total we're going to do a little Panda concat from the data set train now 9:13:49 remember the end of the data set train is part of the data going in and let's just visualize that just a little bit here's our train data let me just put TR for train and it went up to this value here but each one of these values generated a bunch of columns it was 60 across and this value here equals this one and this value here equals this one and this value here equals this one and so we need these top 60 to go into our new data so to find out what we're looking at we're going to go ahead and load up our test data the test data that we didn't process yet and uh real stock price data set test iocation this is the same thing we did when we prepped the data in the first place so let's go ahead and go through this code and we can see we've labeled it part three making the predictions and visualizing the results so the first thing we need to do is go ahead and read the data in from our test CSV you see I've changed the path on it for my computer and uh then we'll call it the real stock price and again we're doing just the one 9:14:50 column here and the values from location so it's all the rows and just the values from these that one location that's the open Stock open let's go ahead and run that so that's loaded in there and then let's go ahead and uh create we have our inputs we're going to create inputs here and this should all look familiar this is the same thing we did before we're going to take our data set total we're going to do a little Panda concap from the data sayate train now remember the end of the data set train is part of the data going in and let's just visualize that just a little bit here's our train data let me just put TR for train and it went up to this value here but each one of these values generated a bunch of column so was 60 across and this value here equals this one and this value here equals this one and this value here equals this one and so we need these top 60 to go into our new data cuz that's part of the next data or it's actually the top 59 so that's with this first setup is over here is we're going in we're doing the real stock price and we're going to just take the data set 9:15:52 test and and we're going to load that in and then the real stock price is our data test. test location so we're just looking at that first uh column the open price and then our data set total we're going to take pandas and we're going to concat and we're going to take our data set train for the open and our data set test open and this is one way you can reference these columns we've referenced them a couple different ways we've referenced them up here with the one two but we know it's labeled as a panda set is open so pandas is great that way lots of Versatility there and we'll go ahead and go back up here and run this there we go and uh you'll notice this is the same as what we did before we have our open data set we pended our two different or concatenated our two data sets together we have our inputs equals data set total length data set total minus length of data set minus test minus 60 values so we're going to run this over all of them and you'll see why this works because normally when you're running your test set versus your training set you run them completely separate but when we graph this you'll see that we're just going to be we'll be 9:16:49 looking at the part that uh we didn't train it with to see how well it graphs and we have our inputs equals inputs. reshapes or reshaping like we did before we're Transforming Our inputs so if you remember from the transform between zero and one and uh finally we want to go ahead and take our X test and we're going to create that X test and for I and range 60 to 80 so here's our X test and we're appending our inputs I to 60 which remember is 0 to 59 and I comma 0 on the other side so it's just the First Column which is our open column and uh once again we take our X test we can convert it to a numpy array we do the same reshape we did before and uh then we get down to the final two lines and here we have something new right here on these last two lines let me just highlight those or or mark them predicted stock price equals regressor do predict X test so we're predicting all the stock including both the training and the testing model here and then we want to take this prediction and we want to inverse the transform so remember we put them between zero and one well that's not going to mean very 9:17:49 much to me to look at a at a float number between 0 and one I want the dollar amounts I want to know what the cash value is and we'll go ahead and run this and you'll see it runs much quicker than the training that's what's so wonderful about these neural networks once you put them together it takes just a second to run the same neural network that took us what a half hour to train ahead and plot the data we're going to plot what we think it's going to be and we're going to plot it against the real data what what the Google stock actually did so let's go ahead and take a look at that in code and let's uh pull this code up so we have our PLT that's our uh oh if you remember from the very beginning let me just go back up to the top we have our matplot library. pyplot as PLT that's where that comes in and we come down here we're going to plot let me get my drawing thing out again we're going to go ahead and PLT is basically kind of like an object it's one of the things that always through me when I'm doing graphs in Python because I always think you have to create an object and then it loads that class in there well in this case PLT is like a canvas you're putting 9:18:48 stuff on so if you've done HTML 5 you'll have the canvas object this is the canvas so we're going to plot the real stock price that's what it actually is and we're going to give that color red so it's going to be in bright red we're going to label it real Google stock price and then we're going to do our predicted stock and we're going to do it in blue and it's going to be labeled predict it and we'll give it a title because it's always nice to give a title to your uh graph especially if you're going to present this to somebody you know to your shareholders in the office and uh the X label is going to be time because it's a Time series and we didn't actually put the actual date and time on here but that's fine we just know they're incremented by time and then of course the Y label is the actual stock price pt. Legend tells us to build the legend on here so that the color red and and real Google stock price show up on there and then the plot shows us that actual graph so let's go ahead and run this and see what that looks like and you can see here we have a nice graph and let's talk just a little bit about this graph before we wrap it up here's our Legend I was telling you about 9:19:49 that's why we have the legend to showed the prices we have our TI and everything and you'll notice on the bottom we have a Time sequence we didn't put the actual time in here now we could have we could have gone ahead and um plotted the X since we know what the the dates are and plotted this to dates but we also know this only the last piece of data that we're looking at so last piece of data which ends somewhere probably around here on the graph I think it's like about 20% of the data probably less than that we have the Google price and the Google price has this little up jump and then down and you'll see that the the actual Google instead of a a turn down here just didn't go up as high and didn't low go uh down so our prediction has the same pattern but the overall value is pretty far off as far as um stock but then again we're only looking at one column we're only looking at the open price we're not looking at how many volumes were traded like I was pointing out earlier we talk about stock just right off the bat there's six columns there's open high low close volume then 9:20:50 there's WEA uh I mean volume shares then there's the adjusted open adjusted High adjusted low adjusted close they have a special formula to predict exactly what it would really be worth based on the value of the stock and then from there there's all kinds of other stuff you can put in here so we're only looking at one small aspect the opening price of the stock and as you can see here we did a pretty good job this curve follows the curve pretty well it has like a you know little jumps on it bins they don't quite match up so this Bend here does not quite match up with that bin there but it's pretty darn close we have the basic shape of it and the prediction isn't too far off and you can imagine that as we add more data in and look at different aspects in the specific domain of stock we should be able to get a better representation each time we drill in deeper of course this took a half hour for my program my computer to train so you can imagine that if I was running it across all those different variables might take a little bit longer to train the data not so good for doing a quick tutorial like this so we're going to 9:21:50 dive right into what is Ross we'll also uh go all the way through this into a couple of tutorials because that's where you really learn a lot is when you roll up your sleeves so we talk about what is carass carass is a highlevel deep learning API written in Python for easy impl implementation of neural networks uses deep learning Frameworks such as tensorflow pytorch Etc as backend to make computation faster and this is really nice because as a programmer there is so much stuff out there and it's evolving so fast it can get confusing and having some kind of high level order in there we can actually view it and easily program these different neural networks uh is really powerful it's really powerful to to um uh have something out really quick and also be able to start testing your models and seeing where you're going so cross works by using complex deep learning Frameworks such as tensor flow pie torch um ml play Etc as a back end 9:22:51 for fast copy ation while providing a userfriendly and easy tolearn front end and you can see here we have the cross API uh specifications and under that you'd have like TF carass for tensor flow thano coras and so on and then you have your tensorflow workflow that this is all sitting on top of and this is like I said it organizes everything the heavy lifting is still done by tensor flow or whatever you know underlying package you put in there and this is really nice because you don't have to um dig as deeply into the heavy-end stuff while still having a very robust package you can get up and running rather quickly and it doesn't distract from the processing time because all the heavy lifting is done by packages like tensor flow this is the organization on top of it so the working principle of carass uh the working principle of caros is cross uses computational graphs to express and evaluate mathematical expressions 9:23:53 you can see here we put it in blue they have the expression U expressing complex problems as a combination of simple mathematical operators uh where we have like the percentage or in this case in Python that's usually your uh left your um remainder or multiplication uh you might have the operator of x uh to the power of3 and it uses useful for calculating derivatives by using uh back propagation so if we're doing with neural networks we send the error back up to figure out how to to change it uh this makes it really easy to do that without really having not banging your head and having to handr write everything it's easier to implement distributed computation and for solving complex problems uh specify input and outputs and make sure all nodes are connected and so this is really nice as you come in through is that um as your layers are going in there you can get some very complicated uh different setups nowadays which we'll look at in just a second and this just makes it really really easy to start 9:24:53 spinning this stuff up and trying out the different models so when we look at carass models uh carass model we have a sequential model sequential model is a linear stack of layers where the previous layer leads into the next layer and this if you've done anything else even like the sklearn with their neural networks and propagation and any of these setups this should look familiar you should have your input layer it goes into your layer one layer two and then to the output layer and it's useful for simple classifier decoder models and you can see down here we have the model equals a coros sequential and this is the actual code you can see how easy it is uh we have a layer that's dense your layer one as an activation they're using the Rao in this particular example and then you have your name layer one layer dense Rao name Layer Two and so forth uh and they just feed right into each other so it's really easy just to stack them as you can see here and it automatically takes care of everything else for you and then there's a functional model and this is 9:25:55 really where things are at this is new make sure you update your Koss or you'll find yourself running this um doing the functional model you'll run into an error code because this is a fairly new release and he uses multi-input and multi-output model the complex model which Forks into two or more branches and you can see here we have our image inputs equals your cross input shape equals 32x 32x 3 you have your your dense layers dense 64 activation reu this should look similar to what you already saw before uh but if you look at the graph on the right it's going to be a lot easier to see what's going on you have two different inputs uh and one way you could think of this is maybe one of those is a small image and one of those is a full-sized image and that feedback goes into you might feed both of them into one Noe because it's looking for one thing and then only into one Noe for the other one and so you can start to get kind of an idea that there's a lot of use for this 9:26:57 kind of split and this kind of setup uh where you have multiple information coming in but the information's very different even though it overlaps and you don't want it to send it through the same neural network um and they're finding that this trains faster and is also has a better result depending on how you split the data up and and how you Fork the models coming down and so in here we do have the two complex uh models coming in uh we have our image inputs which is a 32x 32 by3 or three channels or four if you're having an alpha channel uh you have your dense your layers dense is 64 activation using The Rao very common uh x equals dense inputs X layers dense x64 activation equals Ru X outputs equals layers dense 10 X model equals coros model inputs equals inputs outputs equals outputs name equals nin model uh so we add a little name on there and again this is this kind of split here this is setting us up to um have the 9:27:58 input go into different areas so if you're already looking at carus you probably already have this answer what are neural networks uh but it's always good to get on the same page and for those people who don't fully understand neural networks to dive into them a little bit or do a quick overview neural networks are deep learning algorithms modeled after the human brain they use multiple neurons which are mathematical operations to break down and solve complex maical problems and so just like the neuron one neuron fires in and it fires out to all these other neurons or nodes as we call them and eventually they all come down to your output layer and you can see here we have the really standard graph input layer a hidden layer and an output layer one of the biggest parts of any data processing is your data pre-processing uh so we always have to touch base on that with a neural network like many of these models they're kind of uh when you first start using them they're like a black box you put your 9:28:59 data in you train it and you test it and see how good it was and you have to pre-process that data because bad data in is uh bad outputs so in data pre-processing we will create our own data examples set with carass the data consists of a clinical trial conducted on 2100 patients ranging from ages 13 to 100 with a the patients under 65 and the other half over 65 years of age we want to find the possibility of a patient experiencing side effects due to their age and you can think of this in today's world with uh covid uh what's going to happen on there and we're going to go ahead and do an example of that in our uh life Hands-On like I said most of this you really need to have hands on to understand so let's go ahead and bring up our anaconda and uh open that up and open up a Jupiter notebook for doing the python code in now if you're not familiar with those you can use pretty much any of your uh setups I just like 9:30:00 those for doing demos and uh showing people especially shareholders it really helps it's a nice visual so let me go and flip over to our anaconda and the Anaconda has a lot of cool to tools they just added datal lore and IBM Watson Studio Cloud into the Anaconda framework but we'll be in the Jupiter lab or Jupiter notebook um I'm going to do Jupiter notebook for this because I use the lab for like large projects with multiple pieces because it has multiple tabs where the notebook will work fine for what we're doing and this opens up in our browser window because that's how Jupiter notebook so Jupiter notebook is set to run and we'll go under new create a new Python 3 and uh it creates an Untitled python we'll go ahead and give this a title and we'll just call this uh cross tutorial and let's change that to Capital there we go we going and just rename that and the first thing we want to go ahead and 9:31:01 do is uh get some pre-processing tools involved and so we need to go ahead and import some stuff for that like our numpy do some random number Generation Um I mentioned sklearn or your s kit if you're installing sklearn the sklearn stuff it's a s kit you want to look up that should be a tool of anybody who is uh doing data science if you if you're not if you're not familiar with the sklearn toolkit it's huge uh but there's so many things in there that we always go back to and we want to go ahead and create some train labels and train samples uh for training our data and then just a note of what we're we're actually doing in here uh let me go ahead and change this this is kind of a fun thing you can do we can change the code to markdown and then markdown code is nice for doing examples once you've already built this uh our example data we're 9:32:01 going to do experimental there we go experimental drug was tested on 2100 individuals between 13 to 100 years of age half the participants are under 65 and 95% of participants are under 65 experience no side effects well 95% of participants over 65 um experience side effects so that's kind of where we're starting at um and this is just a real quick example because we're going to do another one with a little bit more uh complicated information uh and so we want to go ahead and generate our setup uh so we want to do for I in range and we want to go ahead and create if you look here we have random integers train the labels of pin so we're just creating some random data uh let me go ahead and just run that and so once we've created our random data and if you if I mean you can certainly ask for a copy of the code from Simply learn they'll send you a 9:33:02 copy of this or you can zoom in on the video and see how we went ahead and did our train samples a pin um and we're just using this I do this kind of stuff all the time I was running a thing on uh the had to do with errors following a bell-shaped curve on uh a standard distribution error and so what do I do I generate the data on a standard distribution eror to see what it looks like and how my code processes it since that was the Baseline I was looking for in this we're just doing uh uh generating random data for our setup on here and uh we could actually go in uh print some of the data up let's just do this print um we'll do [Music] train samples and we'll just do the first um five pieces of data in there to see what that looks like and you can see the first five pieces of data in our train samples is 49 85 41 68 19 just random numbers generated in there that's all 9:34:03 that is uh and we generated significantly more than that um let see 50 up here 1,000 yeah so there's 1,000 here 1,000 numbers we generated and we could also if we wanted to find that out we could do a quick uh print the length of it and so or you could do a shape kind of thing and if you're using numpy although the link for this is just fine and there we go it's actually 2100 like we said in the demo setup in there and then we want to go ahead and take our labels oh that was our train labels we also did samples didn't we uh so we could also print do the same thing oh labels uh um and let's change this to labels and [Music] 9:35:03 labels and run that just to double check and sure enough we have 2100 and they're labeled one Z one0 one0 I guess that's if they have symptoms or not one symptoms uh Z none and so we want to go ahead and take our train labels and we'll convert it into a numpy array in this same thing with our samples and let's go ahead and run that and we also Shuffle uh this is just a neat feature you can do in uh numpy right here put my drawing thing on which I didn't have on earlier um I can take the data and I can Shuffle it uh so we have our so it's it just randomizes it that's all that's doing um we've already randomized it so it's kind of an Overkill it's not really necessary but if you're doing a larger package where the data is coming in and a lot of times it's organized somehow and you want to randomize it just to make sure that that you know the input doesn't follow a certain pattern um that 9:36:03 might create a bias in your model and we go ahead and create a scaler uh scaler range uh minimum Max scaler feature range 0 to one uh then we go ahead and scale the uh scaled train samples we're going to go ahead and fit and transform the data uh so it's nice and scaled and that is the age uh so you can see up here we have 49 85 41 we're just moving that so it's going to be uh between zero and one and so this is true with any of your neural networks you really want to convert the data uh to zero and one otherwise you create a bias uh so if you have like a 100 creates a bias versus the math behind it gets really complicated um if you actually start multiplying stuff there's a lot of multiplication addition going on in there that higher end value will eventually multiply down and it will have a huge bias as to how the model fits it and then it will not fit as well and then one of the fun things we can do in 9:37:03 Jupiter notebook is that if you have a variable you're not doing anything with it it's the last one on the line it will automatically print um and we're just going to look at the first five samples on here it's just going to print the first five samples and you can see here we go point uh 9 1 95. 791 so everything's between zero and one and that just shows us that we scaled it properly and it looks good uh it really helps a lot to do these kind of print UPS halfway through uh you never know what's going to go on there I don't know how many times I've gotten down and found out that the data sent to me that I thought was scaled was not and then I have to go back and track it down and figure it out on there uh so let's go ahead and create our artificial neural network and for doing that this is where we start diving into tensor flow and carass uh tensor 9:38:04 flow if you don't know the history of tensor flow it helps to uh jump into we'll just use Wikipedia be careful don't quote Wikipedia on these things because you get in trouble uh but it's a good place to start uh back in 2011 Google brain built disbelief as a proprietary machine learning setup tensor flow became the open source for it uh so tensorflow was a Google product and then it became uh open sourced and now it's just become probably one of the de factos when it comes for neural networks as far as where we're at uh so when you see the tensorflow setup it it's got like a huge following there are some other setups like um the S kit under the skar has her own little neural network uh but the tensorflow is the most robust one out there right now and carass sitting on top of it makes it a very powerful tool so we can leverage both the carass uh easiness in which we 9:39:04 can build a sequential setup on top of tensor flow and so in here we're going to go ahead and do our input of tensor flow uh and then we have the rest of this is all carass here from number two down uh we're g to import from tensorflow the coros uh connection and then you have your tensorflow cross models import sequential it's a specific kind of model we'll look at that in just a second if you remember from the files that means it goes from one layer to the next layer to the next layer there's no funky splits or anything like that uh and then we have from tensorflow Cross layers we're going to import our activation and our dense layer and we have our Optimizer atom um this this is a big thing to be aware of how you optimize uh your data when you first do it Adams as good as any Adam is usually uh there's a number of Optimizer out there there's about uh there's a couple main ons but Adam is usually 9:40:06 assigned to bigger data uh it works fine usually the lower data does it just fine but atom is probably the mostly used but there are some more out there and depending on what you're doing with your layers your different layers might have different activations on them and then finally down here you'll see um our setup where we want to go ahead and use the metrics and we're going to use the tensorflow cross metrics um for categorical cross centropy uh so we can see how everything performs when we're done that's all that is um a lot of times you'll see us go back and forth between tensor flow and then scikit has a lot of really good metrics also for measuring these things um again it's the end of the you know at the end of the story how good is your model do and we'll go ahead and load all that and then comes the fun part um I actually like to spend hours messing with these things and uh four lines of code you're like ah you're G to spend hours on four lines of code um no we don't spend hours 9:41:08 on four lines of code that's not what we're talking about when I say spend hours on four lines of code uh what we have here I'm going to explain that in just a second we have a model and it's a sequential model if you remember correctly we mentioned the sequential up here here where it goes from one layer to the next and our first layer is going to be your input it's going to be uh what they call D which is um usually it's just D and then you have your input and your activation um how many units are coming in we have 16 uh what's the shape What's the activation and this is where it gets interesting uh because we have in here uh railu on two of these and soft Max activation on one of these there are so many different options for what these mean um and how they function how does the ru how does the softmax function and they do a lot of different things um we're not going to go into the 9:42:10 activations in here that is what really you spend hours doing is looking at these different activations um and just some of it is just a uh almost like you're playing with it like an artist you start getting a fill for like a inverse tangent activation or the tan activation takes up a huge processing amount uh so you don't see it a lot yet it comes up with a better solution especially when you're doing uh when you're analyzing Word documents and you're tokenizing the words and so you'll see the shift from one to the other because you're both trying to build a better model model and if you're working on a huge data set um it'll crash the system it'll just take too long to process um and then you see things like soft Max uh soft Max generates an interesting um setup where a lot of these when you talk about railu oops let me do this uh railu 9:43:13 there we go railu has um a setup where if it's less than zero it's zero and then it goes up um and then you might have what they call lazy uh setup where it has a slight negative to it so that the errors can translate better same thing with softmax it has a slight laziness to it so that errors translate better all these little details make a huge different on your model um so one of the really cool things about data science that I like is you build your uh what they call you build to fail and it's an interesting uh design setup oops I forgot the um end of my code here the concept to build a fail is you want the model as a whole to work so you can test your model out so what you can do uh you can get to the end and you can do your let's see where was it overshot down here you can test your test out the the quality of 9:44:13 your setup on there and see where did I do my tensor flow oh here we go I did it was right above me here we go we start doing your cross entropy and stuff like that is you need a full functional set of code so that when you run it you can then test your model out and say hey it's either this model works better than this model and this is why um and then you can start swapping in these models and so when I say I spend a huge amount of time on pre-processing data is probably 80% of your programming time um well between those two it's like 8020 you'll spend a lot of time on the models one once you get the model down once you get the whole code and the flow down uh set depending on your data your models get more and more robust as you start experimenting with different inputs different data streams and all kinds of things and we can do a simple model summary here uh here's our sequential here's our layer our output our parameter this is one of the nice things 9:45:14 about coros is you just you can see right here here's our sequential one model boom boom boom boom everything's set and clear and easy to read so once we have our model built uh the next thing we're going to want to do is we're want to go ahead and train that model and so the next step is of course model training and when we come in here this a lot of times is just paired with the model because it's so straightforward it's nice to print out the model setup so you can have a tracking but here's our model uh the key word in coros is compile Optimizer atom learning rate another term right there that we're just skipping right over that really becomes the meat of uh the setup is your learning rate uh so whoops I forgot that I had an arrow but I'll just underline it a lot of times the learning rate set to 0.0 uh set to 0.01 uh depending on 9:46:17 what you're doing this learning rate um can over fit and underfit uh so you'd want to look up I know we have a number of tutorials out on overfitting and underfitting that are really worth reading once you get to that point in understanding and we have our loss um sparse categorical cross entropy so this is going to tell carass how far to go until it stops and then we're looking for metrics of accuracy so we'll go ahead and run that and now that we've compiled our model we want to go ahead and uh run it fit it so here's our model fit um we have our scaled train samples our train labels our validation split um in this case we're going to use 10% of the data for validation uh batch size another number you kind of play with not a huge difference as far as how it works but it does affect how long it takes to run and it can also affect the bias a little bit 9:47:18 uh most of the time though batch size is is between 10 to 100 um depending on just how much data you're processing in there we want to go ahead and Shuffle it uh we're going to go through 30 epics and uh put a verbose of two let me just go a and run this and you can see right here here's our epic here's our training um here's our loss now if you remember correctly up here we set the loss let's see where was it um compiled our data there we go loss uh so it's looking at the sparse categorical cross entropy this tells us that as it goes how how how much um how much does the um error go down uh is the best way to look at that and you can see here the lower the number the better it just keeps going down and vice versa accuracy we want let's see where's my accuracy value accuracy at the end uh and you can see 619 69 74 it's going up 9:48:20 we want the accuracy would be ideal if it made it all the way to one but we also the loss is more important because it's a balance um you can have 100% accuracy and your model doesn't work because it's overfitted uh again you won't look up overfitting and underfitting models and we went ahead and went through uh 30 epics it's always fun to kind of watch your code going um to be honest I usually uh uh the first time I run it I'm like Ah that's cool I get to see what it does and after the second time of running it I'm like I like to just not see that and you can repress those of course in your code uh repress the warnings in the printing and so the next step is going to be building a test set and predicting it now uh so here we go we want to go ahead and build our test set and we have uh just like we did our training set a lot of times you just split your your initial set up um but we'll go 9:49:21 ahead and do a separate set on here and this is just what we did above there's no difference as far as um the randomness that we're using to build this set on here uh the only difference is that we already um did our scaler up here well it doesn't matter because the the data is going to be across the same thing but this should just be just transform down here instead of fit transform uh because you don't want to refit your data um on your testing data there we go and now we're just transforming it because you never want to transform the test data um easy mistake to make especially on an example like this where we're not doing um you know we're randomizing the data anyway so it doesn't matter too much because we're not expecting something weird and then we went ahead and do our predictions the whole reason we built the model is we take our model we predict and we're going to do here's our 9:50:23 xcal data batch size 10 verbose and now we have our predictions in here and we could go ahead and do a um oh we'll print predictions and then I guess I could just put down predictions and five so we can look at the first five of the predictions and what we have here is we have our age and uh the prediction on this age versus what what we think it's going to be what what we think is going to going to have symptoms or not and the first thing we notice is that's hard to read because we really want a yes no answer uh so we'll go ahead and just uh round off the predictions using the argmax um the numpy argmax uh for predictions so it just goes to a01 and if you remember this is a Jupiter notebook so I don't have to put the print I can just put in uh rounded predictions and we'll just do the first five and you can see here 0 1 0 0 0 so 9:51:26 that's what the predictions are that we have coming out of this um is no symptoms symptoms no symptoms symptoms no symptoms and just as uh we were talking about at the beginning we want to go ahead and um take a look at this there we go confusion matrixes for accuracy check um most important part when you get down to the end of the story how accurate it is your model before you go and play with the model and see if you can get a better accuracy out of it and for this we'll go ahead and use the pyit um the SK learn metrix uh pyit being where that comes from import confusion Matrix uh some iteration tools and of course a nice map plot library that makes a big difference so it's always nice to um have a nice graph to look at um picture is worth a thousand words um and then we'll go ahead call it CM for confusion Matrix y true equals test labels y predict rounded 9:52:28 predictions and we'll go ahead and load in our cm and I'm not going to spend too much time on the plotting um going over the different plotting code um you can spend uh like whole we have whole tutorials on how to do your different plotting on there uh but we do have here is we're going to do a plot confusion Matrix there's our CM our classes normalized false title confusion Matrix cmap is going to be in blues and you can see here we have uh to the nearest cmap titles all the different pieces whether you put tick marks or not the marks the classes the color bar um so a lot of different information on here as far as how we're doing the printing of the of the confusion Matrix you can also just dump the confusion Matrix um into a caborn and real quick get an output it's worth knowing how to do all this uh when you're doing a presentation to the shareholders you don't want to do this 9:53:29 on the Fly you want to take the time to make it look really nice uh like our guys in the back did and uh let's go ahead and do this forgot to put together our CM plot labels we'll go and run that and then we'll go ahead and call the little the definition for our mapping and you can see here plot confusion Matrix that's our the the little script we just wrote and we're going to dump our data into it um so our confusion Matrix our classes um title confusion Matrix and let's just go ahead and run that and you can see here we have our basic setup uh no side effects 195 had side effects uh 200 no side effects that had side effects so we predicted the 10 of them who actually had side effects and that's pretty good I mean I I don't know about you but you know that's 5% error on this and this is because there's 200 here that's where I get five% is uh divide these both by by 9:54:31 two and you get five out of a 100 uh you can do the same kind of math up here not as quick on the flight it's 15 and 195 not an easily rounded number but you can see here where they have 15 people who predicted to have no uh with the no side effects but had side effects kind of setup on there and these confusion Matrix are so important at the end of the day this is really where where you show uh whatever you're working on comes up and you can actually show them hey this is how good we are or not how messed up it is so this was a uh I spent a lot of time on some of the parts uh but you can see here is really simple uh we did the random generation of data but when we actually built the model coming up here uh here's our model summary and we just have the layers on here that we built with our model on this and then we went ahead and trained it and ran the prediction now we can get a lot more complicated uh let me flip 9:55:32 back on over here because we're going to do another uh demo so that was our basic introduction to it we talked about the uh oops here we go okay so implementing a neural network with carass after creating our samples and labels we need to create our coros neural network model we will be working with a sequential model which has three layers and this is what we did we had our input layer our hidden layers and our output layers and you can see the input layer uh coming in uh was the age Factor we had our hidden layer and then we had the output are you going to have symptoms or not so we're going to go ahead and go with something a little bit more complicated um training our model is a two-step process we first compile our model and then we train it in our training data set uh so we have compiling compiling converts the code into a form of understandable by machine we used the atom in the last example a gradient descent algorithm to optimize a model and then we trained our model which means it let it uh learn on training data uh and I actually had a 9:56:34 little backwards there but this is what we just did is we if you remember from our code we just had let me go back here um here's our model that we created summarized uh we come down here and we compile it so it tells it hey we're ready to build this model and use it uh and then we train it and this is the part where we go ahead and fit our model and and put that information in here and it goes through the training on there and of course we scaled the data which was really important to do and then you saw we did the creating a confusion Matrix with caras um as we are performing classifications on our data we need a confusion Matrix to check the results a confusion Matrix breaks down the various misclassifications as well as correct classifications to get the accuracy um and so you can see here this is what we did with the true positive false positive true negative false negative and that is what we went over let me just scroll down 9:57:34 here on the end we printed it out and you can see we have a nice print out of our confusion Matrix uh with the true positive false positive false negative true negative and so the blue ones uh we want those to be the biggest numbers because those are the better SI side and then uh we have our false predictions on here uh as far as this one so it had no side effects but we predicted let's see no side effects predicting side effects and vice versa if getting your learning started is half the battle what if you could do that for free visit scaleup by simply learn click on the link in the description to know more now uh saving and loading models with carass we're going to dive into a more complicated demo uh and you're gonna say oh well that was a lot of complication before well if you broke it down we randomized some data we created the um carass setup we compiled it we trained it we predicted and we ran our Matrix uh so 9:58:34 we're going to dive into something a lot a little bit more fun is we're going to do a face mask detection with carass uh so we're going to build a carass model to check if a person is wearing a mask or not in real time and this might be important if you're at the front of a store this is something today which is um might be very useful as far as some of our you know making sure people are safe uh and so we're going to look at mask and no mask and let's start with a little bit on the data and so in my data I have with a mask you can see they just have a number of images showing the people in masks and again if you want some of this information uh contact simply learn and they can send you some of the information as far as people with and without masks so you can try it on your own and this is just such a wonderful example of this setup on here so before I dive into the mass detection uh talking about being in the current with h covid and seeing that people are wearing masks this particular example I had to go ahead and update to a python 9:59:37 3.8 version uh it might run in a 37 I'm not sure I I kind of skipped 37 and installed 38 uh so I'll be running in a three python 38 um and then you also want to make sure your tensor flow is up to date because the um they call functional uh layers that's where they split if you remember correctly from back uh oh let's take a look at this remember from here the functional model and a functional layer allows us to feed in the different layers into different you know different nodes into different layers and split them uh very powerful tool very popular right now in the edge of where things are with neural networks and creating a better model so I've upgraded to python 3.8 and let's go ahead and open that up and go through uh our next example which includes uh multiple layers um programming it to recognize whether someone wears a mask or not and then uh saving that model so 10:00:39 we can use it in real time so we're actually almost a full um end to end development of a product here uh of course this is a very simplified version and it'd be a lot more to it you you'd also have to do like uh recognizing whether it's someone's face or not all kinds of other things go into this so let's go ahead and jump into that code and we'll open up a new Python 3 oops Python 3 it's working on it there we go um and then we want to go ahead and train our mask we'll just call this train mask and we want to go ahead and train mask and save it uh so it's it's uh save mask train mask detection not to be confused with masking data a little bit different we're actually talking about a physical mask on your face and then from the cross standpoint we got a lot of imports to do here and I'm not going to dig too deep on the 10:01:40 Imports uh we're just going to go ahead and notice a few of them uh so we have in here oops me go alt D there we go have something to draw with a little bit here we have our uh image processing and the image processing right here me underline that uh deals with how do we bring images in because most images are like a a square grid and then each value in there has three values for the three different colors uh carass and tensorflow do a really good job of uh working with that so you don't have to do all the heavy listing and figuring out what's going to go on uh and we have the mobile net average pooling 2D um this again is how do we deal with the images and pooling them uh dropout's a cool thing worth looking up if you haven't when as you get more and more into carass and tensor flow uh it'll Auto drop out certain notes that way you'll get a better um the notes 10:02:41 just kind of die uh they find that they actually create more of a bias than a help and they also add processing time so they remove them um and then we have our flatten that's where you take that huge array with the three different colors and you find a way to flatten it so it's just a one-dimensional array instead of a 2X two by3 uh DSE input we did that in the other one so that should look a little familiar oops there we go our input um our model again these are things we had on the last one here's our Optimizer with our atom um we have some pre-processing on the input that goes along with bringing in the data in uh more pre-processing with image to array loading the image um this stuff is so nice it looks like a lot of work you have to import all these different modules in here but the truth is is it does everything for you you're not doing a lot of pre-processing you're letting the software do the pre-processing um and we're going to be working with the setting something to 10:03:44 categorical again that's just a conversion from a number to a category uh 01 doesn't really mean anything it's like true false um label binarized the same thing uh we're changing our labels around and then there's our train test split classification report um our IM utilities let me just go ahead and scroll down here Notch for these this is something a little different going on down here this is not part of the uh tensor flow or the SK learn this is the S kit setup and tensor flow above uh the path this is part of um open CV and we'll actually have another tutorial going out with the open CV so if you want to know more about Open CV you'll get a glance on it in uh this software especially the ne the second piece when we reload up the data and hook it up to a video camera we're going to do that on this round um but this is part of the open CV thing and you'll see CV2 is usually how that's referenced um but the 10:04:45 IM utilities has to do with how do you rotate pictures around and stuff like that uh and resize them and then the map plot library for plotting because it's nice to have a graph tells us how good we're doing and then of course our numpy numbers array and just a straight OS access wow so that was a lot of imports uh like I said I'm not going to spend I spent a little time going through them uh but we didn't want to go too much into them and then I'm going to create um some variables that we need to go ahead and initialize we have the learning rate number of epics to train for and the batch size and if you remember correctly we talked about the learning rate uh to the4 .001 um a lot of times it's 0.001 or 0.001 usually it's in that uh variation depending on what you're doing and how many epics and they kind of play with the epics the epics is how many times are we going to go through all the data now I have it as two um the actual setup is for 20 and 20 works great the 10:05:47 reason I have it for two is it takes a long time to process one of the downsides of Jupiter is that Jupiter isolates it to a single kernel so even though I'm on an eight core processor uh with 16 dedicated threads only one thread is running on this no matter what so it doesn't matter uh so it takes a lot longer to run even though um tensor flow really scales up nicely and the batch size is how many pictures do we load at once in process again those are numbers you have to learn to play with depending on your data and what's coming in and the last thing we want to go ahead and do is there's a directory with a data set we're going to run uh and this just has images of masks and not masks and if we go in here you'll see data set um and you have pictures with mass they're just images of people with mass on their face uh and then we have the opposite let me go back up here without masks so it's pretty straightforward 10:06:48 they look kind of a skew because they tried to format them into very similar uh setup on there so they're they're mostly squares you'll see some that are slightly different on here and that's kind of important thing to do on a lot of these data sets get them as close as you can to each other and we'll we actually will run in the in this processing of images up here and the cross uh layers and importing and and dealing with images it does such a wonderful job of converting these and a lot of it we don't have to do a whole lot with uh so you have a couple things going on on there and so uh we're now going to be this is now loading the um images and let me see and we'll go ahead and uh create data and labels here's our um uh here's the features going in which is going to be our pictures and our labels going out and then for categories in our list directory directory and if you remember I just flashed that at you it had uh uh face mask or or no face mask those are the two options and we're 10:07:49 just going to load into that we're going to pin the image itself and the labels so we're just create a huge array uh and you can see right now this could be an issue if you had more data at some point um thankfully I have a a 32 gig hard drive or U Ram even that does you could do with a lot less of that probably under 16 or even eight gigs would easily load all this stuff um and there's a conversion going on in here I told you about how we are going to convert the size of the image so it resizes all the images that way our data is all identical the way it comes in and you can see here with our labels we have without mask without mask without mask uh the other one would be with mask those are the two that we have going in there uh and then we need to change it to the one not hot encoding and this is going to take our um um up here we had was was it labels and data uh we want the lab 10:08:49 uh to be categorical so we're going to take labels and change it to categorical and our labels then equal a categorical list uh we'll run that and again if we do uh labels and we just do the last or the first 10 let's do the last 10 just because um minus 10 to the end there we go just so we can see what the other side looks like we now have one that means they have a mask one Zer one Zer so on uh one being they have a mask and zero no mask and if we did this in Reverse I just realized that this might not make sense if you've never done this before let me run this 01 so 0o is uh do they have a mask on zero do they not have a mask on one so this is the same as what we saw up here without mask 1 equals um the second value is without mask so with masks 10:09:50 without mask uh and that's just a with any of your data processing we can't really a zero if you have a 01 output uh it causes issues as far as training and setting it up so we always want to use a one hot encoder if the values are not actual uh linear value or regression values are not actual numbers if they represent a thing and so now we need to go ahead and do our trainx test Test X train y test y um train split test data and we'll go ahead and make sure it's going to be uh random and we'll take 20% of it for testing and the rest for um setting it up as far as training their model this is something that's become so cool when they're training these Set uh they realize we can augment the data what does augment mean well if I rotate the data around and I zoom in I zoom out I rotate it um share it a little bit flip 10:10:51 it horizontally um fill mode as I do all these different things to the data it um is able to it's kind of like increasing the number of samples I have uh so if I have all these perfect samples what happens when we only have part of the face or the face is tilted sideways or all those little shifts cause a problem if you're doing just a standard set of data so we're going to create an augment and our image data generator um which is going to rotate zoom and do all kinds of cool thing and this is worth looking up this image data generator and all the different features it has um a lot of times I'll the first time through my models I'll leave that out because I want to make sure there's a thing we call build the fail which is just cool to know you build the whole process and then you start adding these different things in uh so you can better train your model and so we go and run this and then we're going to load um and then we need to go ahead and you probably would have gotten an error if you hadn't put this piece in right here um I haven't 10:11:54 run it myself cuz the guys in the back did this uh we take our base model and one of the things we want to do is we want to do a mobile net V2 um and this what we this is a big thing right here include the top equals false a lot of data comes in with a label on the top row uh so we want to make sure that that is not the case uh and then the construction of the head of the model that will be placed on the top top of the base model uh we want to go ahead and set that up and you'll see a warning here I'm kind of ignoring the warning because it has to do with the uh size of the pictures and the weights for input shape um so they'll it'll switch things to defaults to saying hey we're going to Auto shape some of this stuff for you you should be aware of that with this kind of imagery we're already augmenting it by moving it around and flipping it and doing all kinds of things to it uh so that's not a bad thing in this but another data it might be if you're working in a different 10:12:54 domain and so we're going to go back here and we're going to have we have our base model we're going to do our head model equals our base model output um and what we got here is we have an average pooling 2D pool size 77 head model um head model flatten so we're flattening the data uh so this is all processing and flattening the images and the pooling has to do with some of the ways it can process some of the data we'll look at that a little bit when we get down to the lower level on this processing it um and then we have our dents we've already talked a little bit about a d just what you think about and then the head model has a Dropout of 0.5 uh what we can do with a Dropout the Dropout says that we're going to drop out a certain amount of nodes while training uh so when you actually use the model you we use all the notes but this drops certain ones out and it helps stop biases from up forming uh so it's really a cool feature 10:13:56 in here they discovered this a while back uh we have another dense mode and this time we're using soft Max activation lots of different activation options here softmax is a real popular one for a lot of things U so is Ru and you know there's we could do a whole talk on activation formulas uh and why what their different uses are and how they work work when you first start out you'll you'll use mostly the ru and the softmax for a lot of them uh just because they're there some of the basic setups it's a good place to start uh and then we have our model equals model inputs equals Bas model. input outputs equals head model so again we're still building our model here we'll go ahead and run that and then we're going to Loop over all the layers in the base model and freeze them so they will not be updated during the first train process uh so for layer and base model layers layers. trable equals False A lot of times when you go through your data 10:14:58 um you want to kind of jump in part way through um I I'm not sure why in the back they did this for this particular example um but I do this a lot when I'm working with series and and specifically in stock data I want it to iterate through the first set of 30 Data before it does anything um I would have to look deeper to see why they froze it on this particular one and then we're going to compile our model uh so compiling the model atom AIT layer Decay um initial learning rate over epics and we go ahead and compile our loss is going to be the binary cross entropy which we have that print out Optimizer for opt metrics is accuracy same thing we had before not a huge jump as far as um the previous code code and then we go ahead and we've gone through all this and now we need to go ahead and fit our model uh so train the 10:16:00 head of the network print info training head run now I skipped a little time because it you'll see the run time here is um at 80 seconds per epic takes a couple minutes for it to get through on a single kernel one of the things I want you to notice on here while we're while it's finishing the processing is that we have up here our augment going on so any time the train X and train y go in there's some Randomness going on there and is jiggling it around what's going into our setup uh of course we're batch sizing it uh so it's going through whatever we set for the batch values how many we process at a time and then we have the steps per epic uh the train X the batch size validation data here's our uh test Test X and Test Y where we're sending that in uh and this again it's validation one of the important things to know about validation is our um when both our 10:17:01 training data and our test data have about the same accuracy that's when you want to stop that means that our model isn't biased if you have a higher accuracy on your uh testing you know you've trained it and your accuracy is higher on your actual test data than something in there is probably uh has a bias and it's overfitted uh so that's what this is really about right here with the validation data and validation steps so it looks like it's let me go ahead and see if it's done processing looks like we've gone ahead and gone through two epics again you could run this through about 20 with this amount of data and it would give you a nice refined uh model at the end we're going to stop at 2 because I really don't want to sit around all afternoon and I'm running this on a single thread so now that we've done this we're going to need to evaluate our model and see how good it is and to do that we need to go ahead and make our predictions um these are predictions on our test X to see what it thinks are 10:18:03 going to be uh so now it's going to be evaluating the network and then we'll go ahead and go down here and we will need to uh turn the index in remember it's it's either zero or one it's uh 0 one 0 one so you have two outputs uh not wearing uh wearing a mask not wearing a mask and so we need to go ahead and take that argument at the end and change those predictions to a zero or one coming out uh and then to finish that off we want to go ahead and let me just put this right in here and do it all in one shot we want to show a nicely formatted classification report so we can see what that looks like on here and there we have it we have our Precision uh it's 97% with the mask there's our F1 score support without a mask 97% um so that's pretty high high setup on there you know three people are going to sneak into the store who are without a mask and that thinks they have a mask 10:19:03 and there's going to be three people with a mask that's going to flag the person at the front to go oh hey look at this person you might not have a mask um that's if I guess it's a set up in front of a store um so there there you have it and of course one of the other cool things about this is if someone's walking into to the store and you take multiple pictures of them um you know this is just an it it would be a way of flagging and then you can take that average of those pictures and make sure they match or don't match if you're on the back end and this is an important step because we're going to this is just cool I love doing this stuff uh so we're going to go ahead and take our model and we're going to save it uh so model save Mass detector. model we're going to give it a name uh we're going to save the format um in this case we're going to use the H5 format and so this model we just programmed has just been saved uh so now I can load it up in model now they can use it for whatever and then if I get more information uh and we start working with that at some 10:20:05 point I might want to update this model um make a better model and this is true of so many things where I take this model and maybe I'm uh running a prediction on uh making money for a company and as my model model gets better I want to keep updating it and then it's really easy just to push that out to the actual end user uh and here we have a nice graph you can see the training loss and accuracy as we go through the epics uh we only did the you know only shows just the one Epic coming in here but you can see right here as the uh um value loss train accuracy and value accuracy starts switching and they start converging and you'll hear converging this is a convergence they're talking about when they say you're you're um I know when I work in the S kit with sklearn neural networks this is what they're talking about a convergence is our loss and our accuracy come together and also up here and this is why I'd run it more than just two epics as you can 10:21:07 see they still haven't converged all the way uh so that would be a cue for me to keep going but what we want to do is we want to go ahead and create a new Python 3 program and we just did our train mask so now we're going to go ahead and import that and use it and show you in a live action um get a view of uh both myself in the afternoon along with my background of an office which is in the middle still of reconstruction for another month and we'll call this uh mask detector and then we're going to grab a bunch of um a few items coming in uh we have our um mobet V2 import pre-processing input so we're still going to need that um we still have our tensor floral image to array we have our load model that's where most of the 10:22:09 stuff's going on this is our CV2 or open CV again I'm not going to dig too deep into that we're going to flash a little open CV code at you uh and we actually have a tutorial on that coming out um our numpy array our IM utilities which is part of the open CV or CV2 setup uh and then we have of course time and just our operating system so those are the things we're going to go ahead and set up on here and then we're going to create this takes just a moment our module here which is going to do all the heavy lifting uh so we're going to detect and predict a mask we have frame face net Mass net these are going to be generated by our open CV we have our frame coming in and then we want to go ahead and and create a mask around the face it's going to try to detect the face and then set that up so we know what we're going to be processing through our 10:23:12 model um and then there's a frame shape here this is just our height versus width width that's all HW stands for um they've called it blob which is a CV2 DNN blob form image frame so this is reformatting this Frame that's going to be coming in literally from my camera and we'll show you that in a minute that little piece of code that shoot set in here uh and we're going to pass the blob through the network and obtain the face detections uh so faet do. set inport blob detections face net forward print detections shape uh so these is this is what's going on here this is that model we just created we're going to send that in there and I'll show you in a second where that is but it's going to be under face net uh and then we go ahead and initialize our list of faces their corresponding locations and the list of predictions from our face mask Network we're going to Loop over the detections and this is a little bit more 10:24:13 work than you think um as far as looking for different faces what happens if you have a fa a crowd of faces um so We're looping through the detections and the shapes going through here and probability associated with the detection uh here's our confidence of detections we're going to filter out weak detection by ensuring the confidence is greater than the minimum confidence uh so we've said it remember zero to one so 0 five would be our minimal confidence probably is pretty good um and then we're going to put in compute bounding boxes for the object if I'm zipping through this is because we're going to do an open CV and I really want to stick to just the carass part and so I'm I'm just kind of jumping through all this code you can get a copy of this code from Simply learn and take it apart or look for the open CV coming out and we'll create a box uh the box sets it around the image ensure the bounding boxes fall within the dimensions of the 10:25:15 frame uh so we create a box around what going to we hoping is going to be the face extract the face Roi convert it from BGR to RGB Channel again this is an open CV issue not really an issue but it has to do with the order um I don't know how many times I've forgotten to check the order colors we working with open CV because it's all kinds of fun things when red becomes blue and blue becomes red uh then we're going to go ahead and resize it process it frame it uh face frame setup again the face the CBT color we're going to convert it uh we're going to resize it image to array pre-process the input uh pin the face locate face. x. y and x boy that was just a huge amount and I skipped over a ton of it but the bottom line is we're building a box around the face and that box because the open CV does a decent job of finding the face and that box is going to go in there and see hey does this person have a mask on 10:26:16 it uh and and so that's what that's what all this is doing on here and then finally we get down to this where it says predictions equals mass net. predict faces bat size 32 uh so these different images of where we're guessing where the face is are then going to go through generate an array of faces if you will and we're going to look through and say does this face have a mask on it and that's what's going right here is our prediction that's the big thing that we're working for and then we return the locations and the prediction the location just tells where on the picture it is and then the um prediction tells us what it is is it a mask or is it not a mask all right so we've loaded that all up so we're going to load our serialized face detector model from dis um and we have our the path that it was saved in obviously you're going to put it in a different path depending on where you have it or however you want to do it and how you saved it on the last one where we trained it uh then we have our weights 10:27:16 path um and so finally our face net here it is equals CB2 dn. read net uh protot text path weights path and we're going to load that up on here so let me go ahead and run that and then we also need to I'll just put it right down here I always hate separating these things in there um and then we're going to load the actual mass detector model from disk this is the the the model that we saved so let's go ahead and run that on there also so this is pulling in all the different pieces we need for our model and then the next part is we're going to create open up our video uh and this is just kind of fun because it's all part of the open CV video setup and me just put this all in as one there we go uh so we're going to go ahead and open up our video we're going to start it and we're going to run it until we're done and this is where we get some real like kind of live action stuff which is 10:28:16 fun this is what I like working about with images and videos is that when you start working with images and videos it's all like right there in front of you it's Visual and you can see what's going on uh so we're going to start our video streaming this is grabbing our video stream Source zero start uh that means it's C grabbing my main camera I have hooked up um and then you know starting video you're going to print it out here's our video Source equals zero start Loop over the frames from the video stream oops a little redundancy there um let me close I'll just leave it that's how they had it in the code so uh so while true we're going to grab the frame from the threaded video stream and resize it to have the maximum width of 400 pixels so here's our frame we're going to read it uh from our visual uh stream we're going to resize it and then we have a returning remember we returned from the our procedure the 10:29:18 location and the prediction so detect and predict mask we're sending it the frame we're sending it the face net and the mass net so we're sending all the different pieces that say this is what's going through on here and then it returns our location and predictions and then for our box and predictions in the location and predictions um and the box is is again this is an open CV set that says hey this is a box coming in from the location um cuz you have the two different points on there and then we're going to unpack the box and predictions and we're going to go ahead and do mask without a mask equals prediction we're going to create our label no mask we create color if the label equals mask l225 and you know this is going to make a lot more sense when I hit the Run button here uh but we have the probability of the label we're going to display the label and bounding box rectangle on the output frame uh and then we're going to go ahead and show the output from the frame 10:30:18 CV2 IM IM show frame frame and then the key equals CV2 weit key one we're just going to wait till the next one comes through from our feed and we're going to do this until we hit the stop button pretty much so are you ready for this let's see if it works we've distributed our uh our model we've loaded it up into our distributed uh code here we've got it hooked into our camera and we're going to go ahead and run it and there it goes it's going to be running and we can see the data coming down here and we're waiting for the popup and there I am in my office with my funky headset on uh and you can see in the background my unfinished wall and it says up here no mask oh no I don't have a mask on uh I wonder if I cover my mouth what would happen uh you can see my no mask goes down a little bit I wish i' brought a mask into my office it's up at the house but you can see here that this say you know there's a 95 98% chance 10:31:20 that I don't have a mask on and it's true I don't have a mask on right now and this could be distributed this is actually an excellent little piece of script that you could start you know you install somewhere on a a video feed on a on a security camera or something and then you'd have this really neat uh setup saying hey do you have a mask on when you enter a store or public transportation or whatever it is or they're required to wear a mask uh let me goe and stop that now if if you want a copy of this uh code definitely give us a hauler we will be going into open CV and another one so I skipped a lot of the open CV um code in here as far as going into detail really focusing on the carass uh saving the model uploading the model and then processing a streaming video through it so you can see that the model works we actually have this working model that hooks into the video camera which is just pretty cool and a lot of fun so I told you we're going to dive in and 10:32:20 really Roll Up Our Sleeve and do a lot of cating today uh we did the basic uh demo up above for just pulling in a caros and then we went into a cross model uh where we pulled in data to see whether someone was wearing a mask or not so very useful in today's world as far as a fully running application today we are going to talk about meta's new AI development called Lama 3.2 a new family of AI models designed for various tasks and we will also see how to install Lama 3.2 and run it privately the smaller models with 1B and 3B parameters are optimized for the mobile devices handling tasks like text summarization and translation without draining your battery the larger models the 11b and 90b vision models can process both text and image simultaneously making them ideal for complex data analysis and industries like Medical Imaging Lama 3.2 is multilingual out of the box supporting eight languages including English Spanish and Hindi meta 10:33:21 rigorously tested these model across 150 benchmarks and is releasing them as a open source allowing developers to customize them to their needs much like how Linux operates in addition to AI models meta is advancing in augmented reality with its upcoming orian glasses aiming to Overlay digital images onto the real world while these are expected around 2027 their Ray and meta smart glasses are all already popular indicating a strong interest in variable AI gadgets overall meta is pushing the boundaries of AI AR and VR Technologies making them more accessible and customizable for everyone it's an exciting time for Tech Enthusiast as these Innovation shape our digital future so without any further Ado let's jump into the demo part so welcome to this demo part of how to install Lama 3.2 privately on your system locally so first you will okay I will give you this AMA link all this link I will give to you on in the description you can check 10:34:23 so first you I have to install this llama okay you have to install AMA so please select your this OS so my OS is window I have already installed it but again I'm telling you the steps okay you can download it once this setup you install I okay I have already installed so I'm pausing it so once this set up install after that you come into this website okay this is the step two here you can see there are many llama models or you can see llm models mistal is there okay and code gamma is there command R is there Lama 3.1 is there the latest one is Lama 3.2 okay with 1B and 3B models okay the these are the basics on so just click on this so you will get one this command AMA run Lama 3.2 just copy it open your command 10:35:26 prompt or Powershell I'm using Powershell just paste it so this thing I have already installed okay press enter it will take sometimes okay it will off like 2 GB or 4 GB something so it will install so I have already installed so it will see send a message okay so I can write here how are you it will respond me I'm just a language model feeling our emotions like human do these things and all okay I will write exit or I quit okay so leave this okay after this our Lama 3.2 is done okay we have installed but we will launch it with the docker okay so I'm closing this again after that come to this Docker okay this is the doer. you come here download 10:36:29 Docker desktop app so don't download this arm64 as of now because it is in beta version okay download this amd64 okay so it's starting getting download so again my Docker desktop is also installed okay so I will open it directly so after installing it will ask you for the email address and the password to sign in not sign sign up like this so there one option will be there skip so you can skip see I have also skipped that is why it is asking for me sign in okay after skipping this thing will come this dashboard container image volumes build and doer so after this don't close this uh Docker okay just minimize it after that our last step is coming to this website open web UI I will give you the you know links so here 10:37:31 you can just copy this ifama is on your computer Docker on this this just copy this open your Powershell paste this there I have done this process as well and enter okay it's done so when you will open your Docker this will come okay Local Host see last started 5 seconds ago so open this port see after opening that Port this will come sign in to open web UI so here you can sign up okay all 10:38:11 right see after signing up so this is this okay game again you can see we are in Local Host okay you can select a model see 3.1 and 3.2 the latest one okay you can set as default okay default model set here you can ask anything to Lama 3.2 okay any questions okay here are some suggests like okay give me ideas what are the five creating things I could do with my kids art okay press enter so you can see llama 3.2 LS so there are two more models of this 11b and 90b which are multimodels which use for you know big things okay so this is how you can run Lama locally into your systems you can ask anything you can create a scripts okay wait 10:39:13 hello I want to start my Instagram okay I will write I want to build community on Reddit what should I do okay it will give answer so here you can see gave me these thing these five creative things okay it's thinking I guess or something it's doing so this is how you can run Lama 3.2 see it took time I know because we are running locally that's why see building community on Reddit can be rewarding experience for this this is choose the right subreddit and the second thing create high quality content so like so this is 10:40:13 how you can run llama 3.2 okay you can use Lama .2 in this terminal as well but if you will you know run locally deploy with Docker so it will good give you the good UI as well see good UI okay you can sign out settings and these things so what are confusion metrics and why do we need them so in machine learning classification is used to divide data into different categories but after we have clean prepared the data and trained our model how we can tell if the model is performing well so there is where confus metrics come in handy so a confusion metrix is a tool that helps measure the performance of a classifier in detail it goes beyond just giving an overall accuracy score it shows exactly where the model is making mistake and how it is classifying each category for example if there is an imbalance in the data where one category has many more instances than other the model might predict the majority category most of the time and still show 10:41:13 a high accuracy but this wouldn't be helpful because because the model is not accurately predicting the minority categories the confusion matrics help visualize these outcome by showing a table of all the predicted and the actual values allowing you to see where the model is performing well and where it needs Improvement so moving forward let's see confusion matrices parts or how to create 2x two Matrix so we can get four different outcomes when comparing the predict actual values from a classifier the first one is true positive this is when the model correctly predicts a positive outcome you predicted positive and it was actually positive the second one is false positive this happens when the model incorrectly predicts a positive outcome for something that is actually negative or a predictive positive but it was actually negative the third one true negative this is when the model correctly predicts a negative outcome you predicted negative and it was 10:42:14 actually negative and the fourth one for false negative this occurs when the model incorrectly predicts a negative outcome for something that is actually positive Q predictive negative and but it was actually positive so moving forward let's see some matrices of confusion Matrix so this is a graph so we know there are four parts true positive false positive false negative and true negative so this one is true positive this one is false positive this one is false negative this one is true negative so just consider a confusion Matrix made for a classifier that classify People based on whether they speaks English or they speak Spanish okay from this diagram you can see that fine so these are some values TP FN FP TN there's nothing to positive to negative like this just from looking at the metrix the performance of our model is not very clear so to find out how 10:43:14 accurate our model is we use the following matrices the first one is accuracy so this is the formula of accuracy TP means true positive means this 86 value so the accuracy is used to find the portion of correctly classified values it tells us how often of classifier is right it is the sum of all the True Values divided by total values so in this case this is 88% okay 86 + 79 2 positive + 2 negative divide by this it will come 8.23 okay and the second one is precision so Precision is used to calculate the model ability to classify positive value correctly it is the true positive divided by the total number of predicted positive value so this is the formula it is true positive divided by true positive plus false positive so in this case the value will come 87.7% okay and the third one is recall 10:44:15 so recall is used to calculate the model's ability to predict positive values how often does the model predict the currect positive values so it is true positives divided by the total number of actual positive values so in this case the value is 88.9 3 okay and the last one is F1 score so it is the harmonic mean of recall and precision it is useful when you need to take both precision and recall into an account so in this case the value is 88.7% so now let's jump into the demo part and see how we can get these values accuracy precision recall and F1 score from our data set using handson so I'm performing this in Google collab so first I will rename it to confusion Matrix okay so yes let's start so first we will import some basic libraries of python like numpy and the pandas so here I like import numpy 10:45:17 as NP and import pandas as PD why I'm writing this NP and PD because I don't want to write again and again nay and pandas okay this is a short form NP so everyone knows I guess nay and pandas because these are the basic Library so naai is used for mostly linear algebra stuff and pandas is used for the creating data frames data processing or if you want to read files CSP files like that okay yeah so now let's import our data okay so data equals to PD dodore CSV then file name data. CSV so I will upload this file on the description box below you guys can check out from there okay yes so let okay what is saying no such file or directory on data CSV I guess it's 10:46:20 there okay it's not let me import it okay yeah okay now file is there yeah it's working fine so now let's check okay five row yeah so this is our data you can see we have columns wait data do columns okay so we have this 32 columns in our data set ID diagnosis radius means texture means parameter mean so I just simply you know took this data there is nothing related to confusion Matrix and all okay we have to just we can perform you know confusion Matrix to you know any data so head is used for displaying top five rows of of our data set okay if I will write data simply you 10:47:23 can see the full data okay with 33 columns and all the rows right so uh now we don't need this ID and one is this unnamed data because there is no values okay so what I will do I will write data dot drop ID comma okay let me copy it because I don't want any spelling mistake yeah copy paste here okay let me write XIs equals to 1 then comma in place equals to True okay let me run it okay if you will check data do head now I guess there will be no idea all 10:48:25 see there is now no ID and all okay this we remove that LGE column as well uname 32 okay so now what I have to do I will write here data dot diagnosis = to 1 if each equals to M okay SSE zero for each in data dot diagnosis diagnosis yeah so if you can see there is m m m value and there is you know other values as well so for M I'm putting one for for the rest I will put 10:49:26 zero okay let's check again yeah okay you can see it's done replacement is done basically okay so let's check the info of the data so these are some basic uh what to the functions of python so these are the column these are the non Nal values these are the DAT values okay float we have or int we have okay so we have only two t type of data sets so now what we will do we will do normalization okay y equals to data dot diagonosis diagnoses dot values okay okay so here I will write xcore data equals to data do 10:50:27 drop diagnosis comma XIs = to 1 okay here I will write xals to xcore data minus NP dot minimum value and the xcore data and divided by so I put using that formula if you guys know okay divided by NP so I'm basically what I'm finding is accuracy okay I'm using that formula NP Dot Max of X for data then minus NP dot main value then xcore 10:51:33 data okay okay some okay access value is wrong sorry spelling is wrong now fine yes it's fine so now we will do train test split so what is train test split okay so first let me route underscore selection import train test split okay so what is train test split so what we do we you know split our data set into training and the testing mostly we train and the some part of data we test okay so here I will set the ratio so xcore 10:52:35 train comma xcore test comma Yore train comma yore for test equals to [Music] train test split then X comma y comma test underscore size equals to 0.15 means 85% we are on the training and 15% we are on the testing part okay comma random state to 42 okay let me run it it's working fine yeah it's working fine so now let's see the accuracy from the random Forest so I will write from skar 10:53:37 dot emble import random random Forest classifier okay then RF equals to random forest classifier and ncore estimators equals to 100 and here I will set random state to one okay then RF this is short form of random Forest dot fit fit xcore train comma Yore train then print random Forest 10:54:38 score comma RF DOT score then xcore test Comm comma Yore test okay now let's check how much the accuracy so accuracy is coming 95% okay which is very good so now what I will do I will create confus confusion Matrix now Yore prediction equals to RF dot predict X underscore test okay then ycore true equals to Yore test okay so here now let's create the array skarn 10:55:38 dot matrix import confusion Matrix okay then I will write CM short form of confusion metric here I will write y true comma y prediction then confusion Matrix so this is our array okay the true positive and this is false negative and this is false positive like this okay so now let's create the visualization of confusion Matrix okay so here I will write import cbor as SNS and 10:56:39 import M plot lip do pyplot as PL PLT means plot okay the short form F comma ax = to PLT do subplots let's give the figure size equals to 5A 5 then SNS do heat map then confusion Matrix comma n equals to true then comma line width equals to 0.5 comma line color equals to let's give red 10:57:39 then fmt equal to I don't need need any you know uh that decimal values comma ax = to X okay then PL dot X label Yore prediction then PLT dot y Lael then y and _ true add the okay forgot to add this Braes then PLT do show yeah so as you can see this is the Y True Values and this is the Y prediction values okay this is the heat map we have created so this is how you can visualize your confusion Matrix fine 10:58:42 so now what is RNN RNN are a typ type of neural network that are designed to process sequential data they can analyze data with temporal Dimensions such as time series speech and text RN can do this by using a hidden St pass from one time stem to the next the next hidden state is updated at each other time step based on the input and the previous hidden State RNN are able to capture short-term dependencies in sequential data but they struggle with capturing long-term dependencies why the LS STM suit so moving forward let's discuss types of lstm gates so lstm models have three types of gates the input gate the forgate gate and the output gate so let's first discuss the input gate the input gate controls the flow of information into the memory cell deciding what to store the input gate determines which values from the input should be updated in the memory set it 10:59:44 uses a sigmoid activation function to scale the values between zero and one and then applies pointwise multiplication to decide what information to store next is forget gate controls the flow of information out of the memory cell deciding what to discard the forget gate decides what information should be discarded from the memory cell it also uses a zmo activation function to scale the values between 0o and one followed by pointwise multiplication to determine what information to forget the last one is output G controls the flow of information out of the lstms deciding what to use for the output the output gate determines the output of the lstm unit it uses a sigmoid activation function to scale the values from 0 to one then applies point5 multiplication to produce the output of the lstm unit so these G implemented using sigmoid function are trained using back propagation they open and close based on the input and the previous hidden State allowing the lstm to selectively retain or discard 11:00:46 information effectively capturing long-term dependencies so now let's discuss application of lstm lstm models are highly effective and used in various application including video analysis analyzing video frames to identify action object and scenes the second is language simulation tasks like language modeling machine translation and text summarization the third one is time series prediction so predicting future values in a Time series the fourth is voice recognition tasks such as speech to text transcription and command recognition the last one is sentiment analysis classifying text sentiment as positive negative or neural so there are many more examples of LSD so now let's move forward and understand LSM model and how it works with example let's consider the task of predicting the next word in a sentence this is a common application of lstm networks in natural language processing so I will break it down step by step using the analogy of remembering a story and deciding what 11:01:46 comes next based on the context so imagine you are reading a story as you read you need to remember what has happened so far to predict what might happen next so let's illustrate with the simple example sentence the cat sat on the dash so you want to predict the next word which could be mat or roof or something else and lstm network helps this make prediction by remembering important parts of the story and forgetting irrelevant details so now let's dive into step by step step process so step by step exclamation using lstm the first one is reading the story input sequence as you read each word in the word sentence you process it and store relevant information for example the you understand its determiner cat you know it's a noun and the subject of the sentence set indicates the action performed by the subject on preposition indicating the relationship between the cat and the next noun so this sequence diagram showing the words being being read and processed so second comes forget G As 11:02:49 you move through the sentence you might decide that some details are no longer important for instance you might decide that knowing the is less important now that you have get and said so the word forget Gates help discard this less important information so this sequence diagram you can see on the screen showing how relevant information is discarded so the third one input gate when you read on you need to decide how relevant this information is so this sequence diagram in the screen is showing how new information is integrated with the last one okay the fourth one is the cell state memory part so this is like your memory of the story so far it carries the information about the subject cat and the action set on so it updates the new information as you read each word okay the cat set on the retaining the important context so so this sequence diagram showing how the memory is updated with the new information so the last one is output 11:03:50 gate when you need to predict the next word the output gate helps you decide based on the current memory cell state so it uses the context the cat set on the so the predict the next word might be mat because cat and mat are often associated with the context so it can predict anything the cat set on the table or on the sofa anything but Matt why I'm saying Matt because cat and Matt are often associated in the same context so this diagram is showing the prediction of the next word based on the current memory so there are many applications where you can use lstm in predictim Time series or next word in the sentence so by the using of LSM Gates input gate forget gate and output gate and updating the sate the network can predict the next word in a sequence by maintaining relevant context and discarding unnecessary information this step-by-step process allow LSM Network to effectively handle sequence and make 11:04:52 accurate prediction based on the context do you know friends that according to the lending statistics the demand for AI and ml specialist is projected to sege by 40% between 2023 to 2027 and on an average an ml engineer is expected to earn around $133 and $336 per year so if you are an aspiring m engineer and thinking about what Innovative projects you can show in your portfolio then your weight is over cuz in this video I'll be covering eight amazing ml projects that you can Showcase in your resume so guys let's start first with a beginner level project and the first project that we are going to encounter that is home value prediction so guys this project aims to develop a productive model to estimate the value of Residential Properties the model will analyze various features such as location square footage number of bedrooms and bathrooms age of the property and other relevant factors by leveraging historical 11:05:54 Property Data the model will be able to provide accurate home value predictions which can be useful for real estate agents buyers and sellers so guys the programming language that we are going to use all over here will be Python and machine learning libraries that we will be using will be psyit learn tens oflow kiras and for data handling libraries we have H numpy and for visualization we have to use mat plot Li and cbon now what will be the approach for this one guys so guys the first one that we have a data collection so here what is going to happen guys so first you have to collect the historical Property Data from the sources like Zillow retailer.com you can also get database from the public real estate databases like kaggle data sets where you have zero home value prediction ensure that the data set include features like location where you have latitude longitude square footage number of rooms earu property type and previous sales The Next Step that comes is data cleaning you have to handle the missing 11:06:55 values by using imputation techniques or removing incomplete records removing outliers that may skew the models prediction normalize or standardize the data to ensure consistency the third one that we have is feature engineering you have to create new features such as proximity to schools crime rates and access to the public transportation encode categorial variables example property type location using techniques like one hot encoding generate interaction features that capture relationship between existing features the fourth one that we have is model selection use regression models like linear regression random forest cradi and boosting neural networks experiment with different models to identify the best performing mod now in the next phase all you have to do guys is model training and evaluation split the data set into training and test sets train the model on the training set and evaluate their performance on the testing set using metrics like rsme which means root means squared error you can use cross validation to ensure the 11:07:56 model's robustness and avoid overfitting the six oneth that we have all is hyperparameter tuning you can optimize the model's hyper parameter using techniques such as grid search or random search to improve accuracy and if you're looking forward to deploy your model then you can develop a web interface using flask or Jango to allow users to input property features and get predictions you can deploy the model on the cloud platform like AWS for scalability now if we talk about the complexity level of this we all know that it is a beginner level project now let us move on to the one more set that is music genre classification and generation so guys this is also one of the most beginner level project this project aims to develop a system that can classify music tracks into different genres and generate new music composition Within specified genre the goal is to build a model that analyzes audio features to categorize music and uses deep learning techniques to create new music this project introduces Advanced concept of audio processing 11:08:57 deep learning and generative models so guys what will be used in this so we'll have programming language that will be python for audio processing we'll be using librosa for machine learning libraries we'll be using tensorflow k pytorch for data handling libraries we'll be using pandas numpy for visualization we'll be using plot L seon for the data set guys you can use gdan music genre data set or you can get it from free music archive so guys in the first phase we are going to have data collection you can update data sets containing music tracks and their corresponding genre from the sources from gtzan music genre data set and the free music archive ensure that a data set includes diverse genre and substantial number of tracks per genr next we'll go for data pre-processing use library librosa to load and pre-process audio files including feature extraction such as mil frequency sepal coefficients chroma features and spectral contrast you can normalize the extracted features to ensure consistent input for the given models now if we 11:09:59 talk about feature engineering guys you can extract additional features from the audio files such as Tempo beat zero Crossing rate Etc create a feature Matrix that represents the extracted audio features then go for the model selection use conventional neural network or recurrent neural networks for the music genre classification split the data set into training and testing data set now if we talk about model training and evaluation guys then you can train the selected classification model on the training set evaluate the model's performance on the testing set using metrics like accuracy precision recall and F1 score use confusion matrices to understand the classification performance across different genres now if we talk about model selection and training for the music generation what you will do guys you can use the generative adverse Networks or recurrent neural networks such as lstm long short-term memory for music generation train the generative model on the data side to create new music sequences next we have model training and evaluation you can train the generative model on sequences of audio features you can 11:11:00 evaluate the generated music by listening tests and by objective metrics like Inception score or fret audio distance if I talk about hyperparameter tuning guys you can optimize the model you can use the hyper parameters using techniques like grid search or random search to improve per performance if I talk about deployment guys you can deploy these models on cloud platforms like AWS now let us move on to our next project so guys the complexity of this project is at the beginner level now let us move to the intermediate level projects next project that we have all over here is sentiment analysis of Twitter data this project aims to develop a sentiment analysis model that can classifi to each at positive negative or neutral the goal is to analyze public sentiment on various topics or events using natural language techniques so guys what will be used all over here so in this we will have programming language like python okay NLP libraries nltk spacy for machine learning libraries you can use Psy learn tensorflow kiras for data handling 11:12:01 libraries we have pandas dpai for visualization we have matplot lip seon and we can use the API Twitter for data collection now how you're going to work on it guys so guys if I talk about the data collection use the Twitter API to collect tweets based on specific hashtags like keywords or topics extract relevant Fields like tweet text user information timestamp Etc then if I talk about data pre-processing guys clean the Tweet text by removing special characters links mentions hashtags and stop words tokenize the text and perform liiz or stemming to reduce the words to their base form next if we talk about feature engineering guys you convert the clean text Data into numerical representation using tfid F bag of words or word embedding now if I talk about model selection guys you can choose a classification algorithms such as logistic regression na buas or lsdm split the data set into training and testing data set now if I talk about model training and evaluation then you can train the selected model on the 11:13:03 training set evaluate the model's performance on the testing set using metrics like accuracy precision recall and FN score use cross validation to ensure the model's robustness if I talk about hper parameter tuning guys you can optimize the model's hyper parameters using grid search or random search to improve the performance for deployment which can be optional you can deploy your model on AWS for realtime sentiment analysis so if I talk about the complexity level guys it's complexity is intermediate so guys our next project is customer segmentation using km's clustering this project aims to segment customers into distinct groups based on their purchasing behavior and demographic information the objective is to understand customer segments and tailor marketing strategies accordingly so guys what programming languages we'll be using so basically we'll be using python for machine learning libraries we'll have psyit learn for data handing libraries we'll have pandas numpy for visualization libraries we'll have ma plot lab cbon and the data set Source 11:14:03 will be e-commerce transaction data how we are going to work on this one for data collection we can obtain a data set of e-commerce transactions that include customer demographics purchase history and product information next we'll have data pre-processing for data pre-processing we are going to do the cleaning of the data by handling the missing values and outliers then for feature engineering we are going to create features like total purchase amount purchase frequency and recency of the purchases then we are going to proceed for the model selection we can use K me's clustering to segment the customers into distinct groups we can determine the optimal number of clusters using methods like elbow methods or Sout score now if I talk about model training an evaluation you can train the K means model on the process data set you can evaluate the quality of clusters by analyzing intra cluster and intercluster distances next we have the evaluation you can visualize the Clusters using techniques like PCA principal component analysis TSN Etc next we have the 11:15:04 hyperparameter tuning now now you can tune this model and interpret the characteristic of each segment you can develop a Target marketing strategies for each segment based on unique Behavior and preferences now deployment is optional you can develop a dashboard using flask or Jango to visualize customer segments and track marketing campaigns so guys if I talk about the complexity of this project so this is an intermediate level project so guys for data set you can use the kaggle customer segmentation data set which is available at the kles platform now the third intermediate level project that we have all over here is building a chatbot with rasa this project aims to build an intelligent chatboard using rasa framework the chatboard will be capable of understanding user queries and providing appropriate responses making it useful for customer support personal assistance or information retrieval what languages we are going to use so it will be python based we will have the NLP libraries like rasa nltk spacy for 11:16:05 machine learning libraries we are going to have psyit learn tensorflow kiras etc for data handling libraries we are going to use pandas numai so guys this was what we are going to do it and how you can work on this one by collecting the data collect the conversation data and FAQs from the target domain annotate the data to create training examples for the chatbot next comes is data pre-processing clean the text data by removing special characters and normalizing the text you can tokenize and limiti the text to prepare for a training the third one we have the models training you can use rasa's nlus component to train a model for intent recognition and entity extraction you can find the dialogue management policies to handle different conversation flows next guys you can perform the feature engineering and integration you can integrate the rasas nlu and core components to build complete chatbot you can connect the chatbot to messenging platform like Facebook Messenger etc for model selection and testing what you can do guys you can test this chatboard with 11:17:05 various inputs to ensure that it handles the scenarios appropriately and you can also select the right model using this now if I talk about model training guys what you have to do you have to collect the user feedback and conversational logs to continuously work on training the model next similarly you have to retrain the model periodically with the new data to see how it is working so that will be your evaluation now for the hyperparameter tuning what you're going to do guys you have to check in those scenarios where it is able to tune up with those scenarios where it can handle the input appropriately and next is deployment so guys for deploying it you can use AWS so guys for a data set you can use rasa's open source so that's a very good data set for you to proceed so guys if you talk about difficulty of this project this is an intermediate level project now let us move on to the advanced level projects for advanced level projects the first one that comes up to my mind is movie similarity from plot summaries now this project aims to 11:18:07 develop a system that can find out recommended movies similar to a given movie based on their plot summaries by analy ing the textual content of the movie plot summaries the model will identify similarities and suggest movies with similar themes story lines or genres this project introduces beginers to natural language processing text similarly measures and recommendation systems what languages we are going to use guys we'll be using python NLP libraries like nltk Spacey machine learning libraries like psychic learn data handling libraries like pandas numpy visualization we can use numpy and data set Source will be IMDb or kle so guys this process is also involving the data collection then you have to go for data cleaning then feature engineering next model selection the similar process as I have discussed in other projects so you have to also go through the same one next what do you have to do guys similarly what you have to do you have to train the model then evaluate the model then hypertune it and finally 11:19:08 proceed for the deployment so this is overall process of this project try to research on the website a lot like how you can extract it so guys you can use kagle or towards data science to research more about this project now guys if I talk about the difficulty of this project and this is an advanced level project now let us move on to the next one that we have all over here that is image segmentation project for brain tumor prognosis this is a very very amazing project and definitely you can put up on your portfolio basically guys this project aims to develop an image segmentation model to identify and de elimate brain tumors from m& scans the goal is to accurately segment the tumor regions which can Aid in prognosis treatment planning surgical interventions this project introduces intermediate level concepts of computer Visions deep learning and medical image analysis so guys what we'll be using all over here for programming languages we can use Python for deep learning libraries we can use tensorflow kiras pyto for image processing libraries we 11:20:11 can use open CV psyit image for data handing libraries we can use pandas numpy for visualization we can use Matt plot cbon now if I talk about what is a process of developing this project the first step will be same data collection so next step you have to go for data pre-processing third step you have to do the model selection where you can use CNN models for image segmentation task then you go for model selection moving ahead you're going to have the model training and evaluation you have to split the data set into training and validation and testing data sets next proceed for the evaluation phase okay evaluate the model with certain metrics so here I can give you certain idea like you can use dice coefficient intersection over Union or accuracy then go for hyper parameter tuning where you have to optimize the model's hyperparameters you can use grid search or random search as we have discussed and finally you can deploy this model on AWS now guys we have come to the final 11:21:11 project this is also very amazing project guys so the complexity level of this project is advanced level now let us move on to our final project that is the impact of climate change on birds this is a very very amazing project and definitely you can add it on your resume this project aims to analyze the impact of climate change on the bird population and migration patterns by examining various climatic factors and their correlation with bird species data the project seeks to predict how climate change might affect bird behavior and distribution this project we'll introduce you some advanced level Concepts like time series analysis environmental data modeling Etc so guys what programming languages we'll be using for data analysis you can see we'll have pandas numai for machine learning libraries we are going to have psychic learn tensor flow for visualization we are going to have Matt plot plotly for GEOS spatel we are going to have geopandas folium for data source we're going to have public data sets on bird observation and climate data sources from eBird now what will the 11:22:12 process flow for this one guys first you have to proceed for data collection gather bird observation from the data like eBird which provides extensive record of bird sightings okay and for climate data you can collect it from NOA including temperature precipitation and other relevant climatic factors over the time and similar next process will be the data pre-processing then you have to proceed for feature engineering then you have to go for model selection okay moving ahead you have to go for model training then evaluation then hyper parameter tuning and finally you have to deploy the model so research about this project see what models you're going to use suppose I can give you a hint about this you can use time series analysis models like ARA or ml models you can also use random forest or gradient boosting for predicting impact on the bird population so guys use Google exhaustively to research about this project this is also a very amazing project and it's going to give you a lot of idea now if I talk about the complexity level of this it is an 11:23:13 advanced level project project imagine a world where computers can solve problems in seconds problems that would take today's fastest supercomputers thousands of years to crack sounds like a science fiction right well thanks to Quantum Computing we are getting closer to that reality and at the heart of this revolution is Microsoft's mayorana chip but what exactly is mayorana why is it so important and how does it change the future of computing let's break it down Microsoft's Mirana chip is groundbreaking development in Quantum Computing designed to create more stable reliable and scalable quantum computers but to fully understand its importance we need to break it down into three key aspects the science behind Morana particles how Morana particles improve Quantum Computing Microsoft's unique approach with Morana qubits let's dive into each of these at first let's understand the sence behind Mir 11:24:14 particles the name meana comes from an Italian physicist Itor Morana who predicted the existence of meana fans back in 1937 these are unique subatomic particles that are their own antiparticles this means that if two murana particles meet they cancel each other out but why does this actually matter in classical physics every particle has an opposite that is an anti particle for example the electron has a positron so when they Collide they anihilate each other but Muran firmians are special they are both the particle and the anti particle at the same time making them behave differently from anything else we have discovered in nature for decades Muran formans were just a theory however in 2012 scientist of Del University of Technology in the Netherlands along with Microsoft provided the first experimental evidence 11:25:14 of their existence in certain Quantum materials Microsoft then spent over a decade refining This research leading to the creation of the meana based Quantum chip now let's talk about how meana particles improve Quantum Computing quantum computers operate using qubits the quantum version of classical bits traditional qubits like superconducting qubits used by Google and IBM are highly sensitive to their environment even the smallest disturbance like a tiny temperature fluctuation can cause them to lose information and this is called Quantum decoherence Morana qubits however solved this issue by using a principle called topological protection here's how they did it Mirana particles naturally resist external noise and interference information stored in murana based qubits is spread out across multiple locations making it much harder to corrupt this results in lower error ratees and more stable Quantum system 11:26:17 which is crucial for scaling up quantum computers think of it like this imagine writing an important message to a single piece of paper if that paper gets destroyed the message is lost forever but if you spread the message across multiple papers losing one doesn't erase the whole thing and this is how murana qubits protect Quantum information now let's talk about Microsoft's unique approach with mayorana qubits while other companies like Google IBM and intella building quantum computers using superconducting pivots Microsoft took a different path their research team develop topological qubits based on murana fans and this approach is groundbreaking for several reasons first of all it gives more stability unlike superc conducting qubits which need complex error correction mayorana qubits naturally protect themselves from errors it provides better scalability traditional qubits require more enormous amounts of Hardware to correct errors 11:27:17 but meana gbits reduced this need making it easier to scale up quantum computers it also provides higher efficiency Microsoft believes that using mbits will allow them to build a quantum computer with far fewer cubits than other methods what came before meana before meana the quantum Computing world was mostly focused on superconducting cbits companies like IBM Google and Intel have been developing these for years Google's Sky mode processor made headlines in 2019 when it claimed Quantum Supremacy solving a problem that would take a supercomputer thousands of years in just 200 seconds IBM has also made significant process building quantum computers with over 1,000 qubits but despite these achievements quantum computers remained unstable and error prone and Microsoft's murana qubits aim to fix this making Quantum Computing more practical for real world application Microsoft murana is a game 11:28:18 changer it's not just another quantum chip it's a very new way of thinking about Quantum Computing by using topological qubits Microsoft is Paving the way for more stable reliable and scalable Quantum future so what do you think will quantum computers change the world as we know it let us know in the comment section Below in today's video we are diving deep into the world of machine learning interview preparation as we gear up for 2024 it's crucial to be well prepared for the questions that can come your way in any machine learning interview we have compiled 30 essential interview questions and answers thoughtfully categorized into beginner intermediate and advanced levels so let's start with beginner level questions and number one is what is machine learning so machine learning is a subset of artificial intelligence that involves the use of algorithms and statistical models to enable computers to perform task without explicit instructions that is by relying on patterns and interference and now moving 11:29:20 to number second question that is what are the different types of machine learning so the three main types of machine learning are number one is supervised learning and then comes unsupervised learning and then there is reinforcement learning now moving to next question that is third that is what is supervised learning so supervised learning involves training a model on a label data set which means each training example is paired with an output level the model learns to predict the output from the input data now moving to the fourth question that is what is unsupervised learning so unsupervised involve training a model on data that does not have labeled responses the model tries to learn the patterns and the structure from the input data so Guys these are the beginner level questions and now we'll move to the fifth question that is what is reinforcement learning so reinforcement learning is a type of machine learning where an agent learns to make decisions by performing actions and receiving Rewards or penalties the goal is to 11:30:20 maximize the cumulative reward so now moving to the sixth question that is what is a model in machine learning so a model in machine learning is a mathematical representation of a real well process it is trained on data to recognize patterns and make predictions or decisions based on new data so now moving to seventh question that is what is overfitting so overfitting occurs when when a machine learning model performs well on the training data but poorly on new unseen data it indicates that the model has learned the noise and details in the training data instead of the actual patterns so now coming to question number eight that is what is underfitting so underfitting occurs when a machine learning model is too simple to capture the underlying patterns in the data it performs poorly on both the training data and new data now move to the next question that is ninth question and the question is what is a confusion Matrix so confusion Matrix is a table used to evaluate the performance of a classification model it summarizes the 11:31:22 number of correct and incorrect predictions made by the model and that is categorized by each class now moving to the 10th question that is what is cross validation so cross validation is a technique for assessing how the results of a statistical analysis will generalize to an independent data set it involves partitioning the data into subsets training the model on some subsets and validating it on the remaining subsets so this was all about that is the 10th question or the overall 1 to 10 questions for beginner level Now we move to intermediate level and here we'll cover 10 questions so we'll start with 11th question that is what is a Roc curve so Roc that is receiver operating cortic curve it is a graphical representation of a classifier's performance across different thresholds it plots the true positive rate that is tpr against a false positive rate that is fpr now moving to 12th question that 11:32:22 is what is precision and recall so Precision is the ratio of correctly predicted positive observations to the total predicted positives and recall is the ratio of correctly predicted positive observations to all actual positives so the formula is precision equal to tp/ TP + FP and the recall is tp/ TP + FN so now we'll move to the 13th question that is what is the F1 score so the F1 score is the harmonic mean of precision and recall it provides a balance between the two metrics and is useful when you need to balance precision and recall fub1 score is equal to twice into Precision into recall and that is divided by Precision plus Recall now we'll move to 14th question and here we will cover regularization so the question is what is regularization so it is a technique used to prevent overfitting by adding a 11:33:23 penalty to the model's complexity and the common types of regularization include L1 that is lasso and L2 Ridge regularization now we'll move to the 15th question that is what is the bias variance tradeoff so the bias variance trade-off is a fundamental issue in machine learning that involves balancing the error introduced by the model's assumptions and the error due to model complexity so a good model should have low bias and low variance now we'll move to the question number 16 that is what is feature engineering so feature engineering is the process of creating new features or modifying existing ones to improve the performance of a machine learning model it involves techniques like normalization and coding categorical variables and creating interaction terms so now we'll move to question number 17 and that is about gradient descent so the question is what is gradient descent and your answer is gradient descent is an optimization algorithm used to minimize the cost 11:34:25 function in machine learning models and it iteratively adjusts the model parameters in the direction of the steepest Descent of the coast function so with this we'll move to the 18th question and that will cover with the difference between bagging and boosting so the question is what is difference between bagging and boosting and you could answer this with starting with bagging that is bootstrap aggregating that involves training multiple models on different subsets of the data and averaging their predictions then comes boosting that involves training models sequentially with each new model focusing on correcting the errors of the previous ones and then we have the question number 19 that is what is a decision tree so a decision tree is a known parametric supervised learning algorithm used for classification and regression it splits the data into subsets based on the value of input features resulting in a tree like structure of decisions Now we move to question number 20 that is what is a random Forest So Random Forest is an 11:35:27 ensemble learning method that combines multiple decision trees to improve the accuracy and robustness of the model it builds each tree using a random subset of features and data points and then averages their predictions so these were the questions that that are for the intermediate level and these are just the basic questions or I will just say the theoretical questions that can be asked in an interview so be prepared for that now we'll move to the advanced level interview questions and we'll start with question number 21 and here also we'll cover the 10 questions so number one question or that is 21th question and the question is what is a support Vector machine so support Vector machine is a supervised learning algorithm used for classification and regression it finds the optimal hyper plane that maximizes the margin between different classes in the feature space and then comes question number 22 that is what is principal component analysis so principal component analysis is a dimensionality reduction technique that 11:36:29 transforms High dimensional data into a lower dimensional Space by finding the directions that is principal components that maximize the variance in the data and then comes the question number 23 that is what is a neural network so a neural network is a series of algorithms that attempt to recognize underlying relationships in a set of data through a process that mimics the way the human brain operates it consists of layer of interconnected nodes or neurons and then comes the question number 24 that is what is deep learning so deep learning is a subset of machine learning that involves neural networks with many layers that is deep neural networks and it is particularly effective for task like image and speech recognition now I'll move to question number 25 that is what is convolutional neural network that is CNN so we will start the answer by answering the interor that a convolutional neural network is a type of deep learning model specifically designed for processing structured grid 11:37:29 data like images it uses convolutional layers to extract special features or the spal features and patterns from the input data now we'll move to the question number 26 that is what is a recurrent neural network or RN so a recurrent neural network is a type of neural network designed for sequential data and it has connections that form directed Cycles allowing it to maintain a memory of previous inputs and process sequences of data so this was all about question number 26 and now we will cover the question number 27 that is what is the difference between badge gradient descent and stochastic gradient descent so batch gradient descent computes the gradient of the cost function using the entire training data set while stochastic gradient descent that is SGD computes the gradient using only one training example at a time so SGD is faster but no easier now we'll move to question number 28 that is what is Dropout in neural networks so Dropout 11:38:31 is a regularization technique used in neural networks to prevent overfitting and it involves randomly setting a fraction of the neurons to zero during training forcing the network to learn more robust features and now we'll move to question number 29 and that will be about transfer learning and your question is what is transfer learning so we'll answer this to the interviewer by starting that transfer learning is a technique in machine learning where a model developed for one task is reused as the starting point for a model on a second related task it is particularly useful when there is limited data available for the second task now we'll move to the last question and the 30th question so that is what is a generative aders Network that is g so you can start answering this so generative adversor network is a type of deep learning model consisting of two neural networks a generator and a discriminator that are trained simultaneously the generator 11:39:32 creates fake data while the discriminator tries to distinguish between real and fake data leading to the generator producing increasingly realistic data and these questions and answers are are over and these covers a wide range of topics in machine learning and should help prepare for interviews at various levels so that's that one a full course if you have any doubts or question you can ask them in the comment section below our team of experts will reply you as soon as possible thank you and keep learning with simply Lear staying ahead in your career requires continuous learning and upskilling whether you're a student aiming to learn today's top skills or a working professional looking to advance your career we've got you covered explore our impressive catalog of certification programs in cuttingedge domains including data science cloud computing cyber security AI machine learning or digital marketing designed in collaboration with leading universities and top corporations and delivered by 11:40:33 industry experts choose any of our programs and set yourself on the path to Career Success click the link in the description to know more hi there if you like this video subscribe to the simply learn YouTube channel and click here to watch similar videos to nerd up and get certified click here
