Math for Programmers

There are a lot of topics I wish I could have included! Originally, I wanted to write about abstract algebra and functional programming. This is a bit more advanced, and so I decided with my publisher to focus on topics that were more accessible and interesting to a broader audience. Also, there’s not much on probability or statistics in this one – I hope to write another book on “statistics for programmers” soon.

I tried to have at least one application in each chapter – the new math in that chapter is the concept of partial derivatives and the gradient. I think that force fields are a useful example to think of when learning about the gradient, which is why I picked it!

Job titles vary, but some software engineer job postings require more math than others. I would say there are a lot of data science and quantitative finance jobs out there that require math, as well as software eng. jobs at high-tech companies (e.g. aerospace, biotech, hardware)

I wish I could tell you! My publisher (Manning) has a tradition of putting people in old-fashioned dress on their covers. This woman is wearing traditional dress from Finland from a few hundred years ago. I picked this one from three options, because I like her smirk! No idea what she is doing with her hand… but it almost looks to me like she’s gesturing at a blackboard

Maybe not with that title, but yes 🙂 Later this year I’m releasing a book relating abstract math and functional programming, and I think it is the right way for mathematicians to learn programming!

Hmm… I don’t know much about computer vision in general, so I can’t say for sure. I bet many of the topics in the book are used in computer vision. And, the last chapter is on image recognition using neural networks!

That sounds great !🙂 Will give it a try and thank you for answering.

I would definitely add some content on probability and statistics! I think it is increasingly important math for applications, and I’m sad it didn’t fit in this book. It’s also not my area of expertise. I think most of the big topics in the current book are good, but if I could do it again, I might trim down some of the les useful sections and subsections (not thinking of any off the top of my head, but there are surely some!)

I agree that statistics and probability are very important. I also think that too many rely on them without full understanding. Do you think machine learning engineers, data scientists, and AI practitioners in general need to focus more on math?

I suppose most engineers, data scientists, etc. know the math they “need” to do their jobs, but I suspect there is room for most people to think more deeply about the math they use. In my book, I’ve tried to emphasize the philosophy behind each topic, as in why it is the way it is, and how it fits into the grand scheme of things. This perspective, I think, is similar to a software engineer taking the time to understand how the compiler of their favorite language works. It gives a deeper perspective on how your tools work.

I don’t, but I am trying to convince one of my friends to write such a book! One of the challenges is including enough relevant example to make the book interesting. Will keep you posted!

Alexey Grigorev good question! I think Math for Programmers has a lot of math necessary for ML – namely linear algebra and multivariable calculus. The main topic not included in my book which is necessary for ML is probability. I suggest buying my book (obviously 😇 ) and taking Andrew Ng’s free coursera course on ML . If you work thorough the exercises in my book and this ML course, you should be prepped to do a deeper dive in any area of ML you want

Thanks! Andrew Ng’s course is awesome!
In your opinion, how a programmer should approach learning probability and stats? You mentioned you might write a sequel for your current book - I guess you already thought about this question

I would say, try to learn whatever you want to do in ML or elsewhere, and read about anything unfamiliar. A lot of what I wrote about in Math for Programmers is stuff I learned for some application (e.g. Physics). having a goal/application in mind is a great way to motivate yourself to learn something, and focus on the most important topics.

I wish I had a good enough 50,000ft view of probability to give more specific advice. This is part of why I’d like to research and put together a book on it.

Makes sense, thank you!

I am kind of a novice, but I think gradient descent is very important!

It is good to think of learning as an optimization problem, and gradient descent is one optimization algorithm you can use pretty widely

I agree that it is important to learn what an optimization problem is and to frame machine learning as one. But it seems like the focus of machine learning is on what to optimize, rather than how.
To me gradient descend feels kind of “academic”. I imagine that not many ever write their own optimizer for machine learning. Probably for most people reading two nontechnical paragraphs explaining the intuition behind gradient descent will be more than enough.

Yeah, I guess it’s a matter of opinion. I think deriving the backpropagation formulas for a MLP (using gradient descent) and implementing them is a worthwhile exercise, because it demystifies machine learning. I totally agree that ML in practice is more about deciding what to optimize.

I don’t know that book, but presumably the difference is that it covers stats while my book does not 🙂

Glad you liked it! I don’t know much about geospatial algorithms, but sounds like it could be a cool application for 2D and 3D geometry. Can you recommend any good references for me to get started?

Ben Wilson do you mean projection algorithms? or stuff like spatial indexes, spatial search and so on?

great circles intersection, polygonal collision, distance measurements with z-components, et. al. I think that with your presentation style you could write an effective primer to basically GIS ( 🙂 ) http://wiki.gis.com/wiki/index.php/New_to_GIS#Related_links

nice :)) I used to do a lot of spatial stuff (postgis mainly but also a bit of python). Would be nice to have a deeper dive into some of the maths definitely

I think the fear comes from school/university when teachers just force math down our throat without really explaining why we should learn it

this may help to manage fear

Alexey Grigorev same problem goes for me

It was the same for me, but if you take your time to break a complicated formula to parts and dig into each of them, then after a couple of iterations it won’t be that scary

I usually learn difficult topics by finding good explanations

Roman G I have found there is a delicate tradeoff between lowering inhibition and keeping ones mental faculties intact with the method you propose 🙂

This one? https://xkcd.com/323/

I like this question because it is a problem I still have – sometimes I find myself looking at a page in a math textbook and thinking to myself this looks hopelessly complex.

Alexey Grigorev yes exactly!

I would say, start by writing down the formulas you do know or drawing diagrams (related to the topic), without looking at a book or other resources. That should show you where the holes in your understanding are. Then go back to the book and try to fill these holes as needed. Over time, you should get some more confidence!

Makes sense, thank you!

I would say yes, I believe that algorithms are basically about breaking mathematical constructs into simple step by step processes or approximations, at school too much emphasis is given to paper math, I think more emphasis should be put into computational thinking in all engineering degrees not only computer science.

A lot of ideas transfer directly from math to computer programming: logic, functions, sets, etc. I’m a strong believer that math skills help you think more rigorously about software (of any kind) that you build

Yes, I would change some things! I think math classrooms should be flipped, meaning record lectures so students can consume the content at their own pace, independently. Then save class time for exploration and problem solving. I would also say that most people who take math in college are not going to become mathematicians, so there should be a greater emphasis on applied math and using technology to solve math problems (e.g. Python or even Excel)

Nice! How would you motivate students to watch the content at their own pace before the classes?

That’s a problem I also have when teaching adults =)

Obviously some students are more self-motivated than others, but I think this would increase the quality of the learning experience and have most people more engaged overall.

Yes, makes sense! Thank you!