Hello world, it’s Siraj and I’m going to show you how I read research papers and give you some additional tips on how to consume them more efficiently. Reading research papers is an art, whether the topic is machine learning or cryptography, distributed consensus or networking, in order to truly have an educated opinion on a particular topic in computer science you’ve got to get yourself acquainted with current research in that subfield. It’s easy to agree with a claim if it’s got enough hype behind it but being critical and balanced in your assessment is a skill that can be learned by researchers as well as non- researchers. PhD students and researchers are taught how to do this in grad school but you too can learn how to do this. It just takes patience and practice and coffee, lots of coffee every single week. I read around 10 to 20 research papers in order to keep up with the field and I’ve gotten better at it over time and I don’t have any graduate degree. I’m just a guy who really loves this stuff and I teach myself everything using our new collective University- the Internet. One of my favorite resources to find papers on machine learning by various researchers is the machine learning subreddit. People post papers they find interesting every day by a number of different researchers and they’ve also got this cool weekly what are you reading thread where people post the papers of researchers that interest them the most currently. Additionally there is this web app called archived sanity com created by Andrey Karpov II which basically goes through archive and finds the papers by researchers that are most relevant. You can filter them by what interests you, by which ones are most popular researchers or by the ones that are most cited by researchers. Lately Google and Deepmind respectively publish their work on their websites for easy access to a no of papers by different researchers. There are of course journals like Nature too where you can find some top papers by researchers easily. The pace of research is accelerating in machine learning because of a few reasons not including Smith you in academia and in the public sphere the democratization of data computing power education and algorithms are all steadily happening over the internet. Because of this more people and researchers are able to make their own insights into this field in the industry. The big tech companies profit more when their own teams of researchers discover new machine learning methods. So there’s this race among researchers to create faster, more intelligent algorithms. All that is to say that there are a lot of papers by researchers that you could be reading right now. So how are you supposed to know what to read well? What I found is that every week there are maybe two or three papers by researchers that are getting the most attention in machine learning and the tools I’ve mentioned helped me find them and read them. But most of my reading is a result of me having a goal; that goal could be to learn more about activation functions or perhaps probabilistic models that use attention mechanisms. Once I’ve got that goal it makes it much easier to create a reading strategy that points towards that goal. Just being a good math heavy machine learning paper reader is not a goal to aspire to. Your stamina is more of a function of human motivation which is a function of the goals you’re trying to accomplish. I found that I can crush through and understand the most difficult papers much more when I have a real reason to do so. So let’s take the landmark paper by a researchers friend of mine, Ian good fellow, on generative adversarial networks as an example. There is a lot in this paper. He synthesizes some ideas here that made Yamla kun say that this concept was the coolest idea in deep learning in the last ten years. The way I read papers is by performing a three pass approach. On the first pass I’ll just skim through the paper to get its meaning. I’ll first read the title, if the title sounds interesting and relevant generative adversarial networks I’ll read the abstract. The abstract acts as a short standalone summary of the work of the paper that people can use as an overview. If the abstract is compelling, an adversarial process between two neural networks that were temples a game all right this is lit then I’ll skim through the rest of the paper, by that I mean I’ll carefully read the introduction then read the section and subsection headings but ignore everything else mainly ignore the math. I never read the math on the first pass I’ll read the conclusion at the end and maybe glance over the references, mentally ticking off the ones I’ve already read if there are any. I just assume the math by researchers is correct on the first pass. My goal for this first pass is to just be able to understand the aims of the researchers, what are the papers main contributions here, what problems does it attempt to solve, is this a paper I’m actually interested in reading more than once. Then I’ll go back to see what other researchers are saying about this paper and compare my initial observations to theirs. Basically the aim of this first pass is to ensure that its worth my time to continue analyzing this paper live short and there are too many things to read. If it does pique my interest then I’ll reread it a second time on the second pass. I’ll read it again this time more critically and I’ll also take notes as I go. I’ll actually read all the English text and I’ll try to get a high level understanding of the math by researchers that’s happening in the paper. So it’s a minimax game that looks to optimize a Nash equilibrium okay.
Various great research journals such as Global Research Letters are a great option and way to help you look up research papers with the help of which you will be able to read the papers and understand the importance of reading research papers on machine learning. Here, you will find a number of various research papers on machine learning that are provided and made available to you in the journal.
You can very easily find the paper you are looking for here, which will help you with your own research work and understanding of how to read research papers in machine learning. With access to so many amazing research papers about machine learning, you can practice and learn how to read research papers about machine learning and its importance.
I kind of get that, eventually the generator Network creates fake samples that are indistinguishable from the real thing by real researchers so the discriminator is powerless. Cool I’ll read the figure descriptions any plots and graphs that are available and try to understand the algorithm by researchers at a high level. A lot of times the author and researchers will break down an equation by factoring it out. I avoid trying to analyze this on the second pass. I see that it’s using a loss function called the kullback-leibler divergence, never heard of that one but I do get the concept of minimizing a loss function when I read the experiments by researchers. I’ll try to evaluate the results are they repeatable. Are the findings well supported by evidence? Once I’ve done that hopefully there is some associated code with the repository available on Github. I’ll download the code and start reading it myself. I’ll try to compile and run the code locally to replicate the results as well. Usually comments in the code help further my understanding. I’ll also look for any additional resources on the web that help further explain the text articles summaries tutorials usually a popular paper will have a breakdown that someone else has done online that will help drive the key points home for me. After this second pass I’ll have a Jupiter notebook full of notes and associated helper images since I teach this stuff on YouTube. Teaching is really the best way to fully understand any topic. When it comes to the third pass it’s all about the math. My focus on the third pass is to really understand every detail of the math. I might just use a pen and paper and break down the equations in the researchers paper myself. I’ll use Wikipedia to help me understand any of the more formal math concepts, fully alike the KL divergence and if I’m feeling really ambitious I’ll try to replicate the paper programmatically using the hyper parameter settings and equations that it describes. After all of this I’ll feel confident enough to discuss it with other people. Greeting papers is not easy and nobody can read long manipulations of complicated equations fast. The key is to never give up. Turn your frustrations into fuel to get better. You will understand the papers by various researchers, you will master this subject, and you will become awesome at this. It gets easier every time as you build your merkel dag of knowledge. See what I did there, if you don’t get a math concept guess what Khan Academy will teach you- anything you need to know for free and lastly, do not hesitate to ask for help. There are study groups and communities online that are centered around the latest research in machine learning by researchers that you can post your questions to. Don’t be afraid to reach out to researchers as well. You’re actually doing the researchers a favor by having them explain to you in terms you understand. All scientists and researchers need more experience translating complex topics. I’ve got lots of great links for you in the description and I hope you found this video useful. if you want to learn more about machine learning, AI and block technology, hit the subscribe button and for now I’ve got to reread the capsule Network paper so, thanks for watching.