Convex Optimization

As you begin your journey into machine learning, understand that the algorithms that you use stand on very firm ground, mostly in optimization. Most optimization problems can in general be thought of as solution finding in some Rn. Now turns out in certain classes of optimization, we can find some form of global optimum, and this class is the class of convex sets. Optimization is used everywhere, and all of us have used it already. A simple thing like f'(x) =0 is something that we have already used from school days. But the problem is twofold. Recognizing that a function in convex, for a general f, and then trying to solve f' = 0 in cases where f is not conducive to exact solutions. -- This is a great first book for someone looking to enter the world of machine learning through optimization. This is another approach apart the statistical side (which is well covered in ESL by Hastie and Tibshiriani). Several fundamental things in machine learning like SVMs and gradient descent are based on the concepts learnt from optimization. One of the strengths of this book is that it doesn't jump into the solution methods or how to solve such problems until the problem is well understood. Spending more time in the problem space than the solution space let's the reader know why and when to apply the solutions suggested. The algorithms form the third (and last) part of the book. In a book like Fletcher, newtons method would be in the first chapter after the introduction. This approach too aids the new entrant to the field. I find this book a slightly easier read than the one by Bersetkas. That could be a good second book, before you move on to other topics based on your interest. If you are interested in finding solutions in Rn for general cases of f (say non convex), core optimization books like Luenberger or Fletcher may be recommendable, especially for numerical optimization enthusiasts. But these are only appreciated after a first pass through the subject. Nocedal and Wright is a wonderful book for someone with exposure to optimization. -- As always I find the prints from Cambridge Press extremely readable and a pleasure to hold. I would recommend buying this book even though you may get online prints. -- Problems in this book are hard. You may have to struggle a bit to solve the problems completely. This might affect your choice of whether to use this book as a textbook for convex optimization. -- *Important*: Supplement the book by the highly recommended set of video lectures by the same Author (Boyd) on convex optimization available online. His conversational tone, and casual dropping of profound statements makes the video lectures some of the best I have seen. -- Prerequisites: To appreciate the book, you need to have understood linear algebra (say atleast at level of Strang) as well as calculus (Joydeep Dutta recommended for this) -- Overall: Recommended book in the library of every machine learning enthusiast. Not a must have, but almost there.

Classic book based on the iconic course taught at Stanford EE. Delivered in perfect condition. Thanks Amazon for the excellent service !!!

paper quality too bad . should be of rs 500 for the quality got it for 2500 from campus book house

I was afraid it might be an international version as others reviewed, but it was not. Glad to buy it on sale.

Arrived on time, in excellent condition as described.

this book is a very good monograph on convex optimization, which i would recomend anyone who only wants to read one book on the topic. contains theory and some of the most important aspects of algorithms for convex optimization problems.

self-containedの一冊．世界から評価されている安心の一冊． 読者は気力のあるもののみ．その他は拒む． 普通例示は一般化された概念に対して具体的な例を挙げて理解を補うものであるが，この本に限っては例示で付随した性質，定理が学べるため，とても重要な役割になっている．実際に数理最適化の研究で必要な定理を証明していることもあるので，例を飛ばして読むのはかなり勿体無い気がする．よって，例が難しく感じるのは自然なことだと思う． 1周で終わらせることはできないので，何度も読む本．無限のスルメ本ともいえる．

This review is written from the perspective of a mathematically capable outsider to the field of Operations Research and Optimization that does research in statistics & machine learning. I have found that it's very difficult to figure out how to self-study these topics in general, so I'm going to include context & other resources I have found useful to highlight the importance and quality of this book. In my opinion, the landscape of OR and Optimization have to be understood to really appreciate it. When I initially picked up Boyd & Vanderberghe, I got very frustrated with it. I would recommend going through introductory optimization material first (I went through "Managerial Decision Modeling with Spreadsheets" by Balakrishnan et al, which is excellent), and be comfortable with the basics of linear algebra and calculus, otherwise this book will seem more inaccessible than it actually is. I have also heard that Stephen Boyd's lectures on YouTube are an excellent supplement to the book and make the introduction far more gentle. This book is a field manual for learning to recognize convex problems "in the wild": meaning, given an optimization problem, how do we make it easier to solve by making one or more parts of the problem convex? This process is known as "convex relaxation" and is one of the fundamental skills required for research in topics like statistics & machine learning, and probably many other fields as well. Because of this focus, the book strikes a very important middle ground between the highly applied side of case studies and model building, and the highly theoretical side of algorithms and convex analysis. It provides just enough theory to be able to work with problems, and just enough intuition on algorithms to know what's going on behind the scenes. I don't know why, but most treatments I have seen of Optimization/Operations Research is presented with too much of a focus on either theory/algorithms, or case studies with barely any focus on the former. After reading this text and doing the exercises (for which the solutions are freely available online), the reader can expect to be able to formulate optimization problems, relax those problems to be more easily solvable, input the problems to an optimization solver like CVX (Matlab) or CVXPY (CVX, but in Python) and have an idea of what algorithms the solver will use to solve the problem. From what I have seen so far, I believe this text alone is sufficient for a good initial background to do statistics & machine learning research. However, I don't believe it's enough for people who want to focus on the difficulties of Operations Research model building, writing optimization algorithms, or on theoretical topics like existence of solutions, convergence proofs for algorithms, etc. For discussion on the challenges associated with building big optimization models, I recommend H. Paul Williams's "Model Building in Mathematical Programming" (2013)--multiple real-world problems, their model formulations, and solutions are presented, and it's just amazing practice for a budding analyst. I think the introductory chapters are definitely lacking though, and I recommend picking up "Managerial Decision Modeling with Spreadsheets", as I mentioned above. For learning how to write numerical optimization algorithms, it is hard to go wrong with Nocedal's and Wright's "Numerical Optimization" (2006). The presentation is, like this book, crystal clear, but again requires the reader to have a good grasp on the basics of optimization, linear algebra, and calculus. The entire focus of the book is on algorithms to solve various optimization problems (not just convex problems!), but because of its age, it will not contain material on some of the newer algorithms that have found use. I would recommend looking through (free) online courses such as Ryan Tibshirani's "Machine Learning 10-725", Stephen Boyd's EE364A/EE364B sequence, and Emmanuel Candes's Math 301. Unfortunately, I am not the right person to ask for book recommendations to learn the theory, but I have heard good things about books by Nesterov, Bertsekas, and Rockafellar. These are books that really demand the reader to have a background in mathematical analysis. From my limited searching, Bertsekas's books have solution manuals, have good geometric descriptions of the concepts, and are self-contained. The books "Convex Optimization Theory" (2009), "Convex Optimization Algorithms" (2015), and "Convex Analysis & Optimization" (2003) in particular might be a good place to start.