Questions and Answers
Hi Paul, my questions are:
Are there topics that you really want to cover in the book, but decided against in the end?
There are a lot of topics I wish I could have included! Originally, I wanted to write about abstract algebra and functional programming. This is a bit more advanced, and so I decided with my publisher to focus on topics that were more accessible and interesting to a broader audience. Also, there’s not much on probability or statistics in this one – I hope to write another book on “statistics for programmers” soon.
Most of the topics in the list of contents seems more or less fairly general, except the chapter on gravity simulation– what made you add that particular topic?
I tried to have at least one application in each chapter – the new math in that chapter is the concept of partial derivatives and the gradient. I think that force fields are a useful example to think of when learning about the gradient, which is why I picked it!
In the intro, you mentioned that you wrote it partly with the idea of ‘scientific software engineer’ in mind– it’s not a job description I see very often, outside of universities, where do these jobs come up? And what sort of skills do they usually look for? (ie do they assume you’ll have a background in the science as well?)
Job titles vary, but some software engineer job postings require more math than others. I would say there are a lot of data science and quantitative finance jobs out there that require math, as well as software eng. jobs at high-tech companies (e.g. aerospace, biotech, hardware)
Hi Poul,
The book’s cover is interesting.
But particularly the lady’s hand sign is prominent.
İs there a meaning in it or is it aimless?
I wish I could tell you! My publisher (Manning) has a tradition of putting people in old-fashioned dress on their covers. This woman is wearing traditional dress from Finland from a few hundred years ago. I picked this one from three options, because I like her smirk! No idea what she is doing with her hand… but it almost looks to me like she’s gesturing at a blackboard
Hey Paul Orland , great book 👏🏼, this is definitely something I want to expand my understanding in some more.
I like that you’ve added exercises at the end of the topics; How did you get your ideas for the exercises you listed? Are they short questions to test understanding or small practical project styled etc?
Glad you’re enjoying it so far! Good question. I’ve designed most of the exercises to be doable in 1-15 minutes
They are supposed to be relatively quick checks of your understanding, to help you make sure you are keeping up with the material. Then, if one takes longer than a few minutes to figure out, it’s a good sign you should reread the chapter and think through the topic again.
Anything more involved is marked as a “mini project”, indicating that it will take what you’ve learned and go a bit beyond the material introduced in the text.
The way I come up with these is roughly: I think of some small example I could work through in a paragraph, and rather than explaining it, I try to give a carefully worded hint so you can basically connect the dots and figure it out yourself.
Will there be a book called “Programming for mathematicians”? Asking for a friend.
Maybe not with that title, but yes 🙂 Later this year I’m releasing a book relating abstract math and functional programming, and I think it is the right way for mathematicians to learn programming!
Hi Paul Orland, glad to converse with you. Are you covering the mathematical fundamentals needed for computer vision ( I saw you were covering 3D graphics) such as various image transforms. Thanks.
Hmm… I don’t know much about computer vision in general, so I can’t say for sure. I bet many of the topics in the book are used in computer vision. And, the last chapter is on image recognition using neural networks!
That sounds great !🙂 Will give it a try and thank you for answering.
Hello, Paul Orland! Thanks for doing this. My question(s): if you were going to expand your book, what would you add? What would you expand? What would you cut?
I would definitely add some content on probability and statistics! I think it is increasingly important math for applications, and I’m sad it didn’t fit in this book. It’s also not my area of expertise. I think most of the big topics in the current book are good, but if I could do it again, I might trim down some of the les useful sections and subsections (not thinking of any off the top of my head, but there are surely some!)
I agree that statistics and probability are very important. I also think that too many rely on them without full understanding. Do you think machine learning engineers, data scientists, and AI practitioners in general need to focus more on math?
I suppose most engineers, data scientists, etc. know the math they “need” to do their jobs, but I suspect there is room for most people to think more deeply about the math they use. In my book, I’ve tried to emphasize the philosophy behind each topic, as in why it is the way it is, and how it fits into the grand scheme of things. This perspective, I think, is similar to a software engineer taking the time to understand how the compiler of their favorite language works. It gives a deeper perspective on how your tools work.
Hi Paul Orland do you plan to have a separate book on the lines of Maths for Data Scientists? Slightly more deeper and covering topics on numerical optimization,infinite series etc but not in the typical classical math textbook format.
I don’t, but I am trying to convince one of my friends to write such a book! One of the challenges is including enough relevant example to make the book interesting. Will keep you posted!
Hi Paul!
Which learning plan would you recommend for software engineers who want to get into machine learning? What kind of math do they need to focus on - to make the learning most effective? And would your book be enough to cover the math fundamentals?
Alexey Grigorev good question! I think Math for Programmers has a lot of math necessary for ML – namely linear algebra and multivariable calculus. The main topic not included in my book which is necessary for ML is probability. I suggest buying my book (obviously 😇 ) and taking Andrew Ng’s free coursera course on ML . If you work thorough the exercises in my book and this ML course, you should be prepped to do a deeper dive in any area of ML you want
Thanks! Andrew Ng’s course is awesome!
In your opinion, how a programmer should approach learning probability and stats? You mentioned you might write a sequel for your current book - I guess you already thought about this question
I would say, try to learn whatever you want to do in ML or elsewhere, and read about anything unfamiliar. A lot of what I wrote about in Math for Programmers is stuff I learned for some application (e.g. Physics). having a goal/application in mind is a great way to motivate yourself to learn something, and focus on the most important topics.
I wish I had a good enough 50,000ft view of probability to give more specific advice. This is part of why I’d like to research and put together a book on it.
Makes sense, thank you!
What’s your favorite example, that shows importance of understanding math behind ML? Most of the examples I can think of don’t really justify a deep dive…
I am kind of a novice, but I think gradient descent is very important!
It is good to think of learning as an optimization problem, and gradient descent is one optimization algorithm you can use pretty widely
I agree that it is important to learn what an optimization problem is and to frame machine learning as one. But it seems like the focus of machine learning is on what to optimize, rather than how.
To me gradient descend feels kind of “academic”. I imagine that not many ever write their own optimizer for machine learning. Probably for most people reading two nontechnical paragraphs explaining the intuition behind gradient descent will be more than enough.
Yeah, I guess it’s a matter of opinion. I think deriving the backpropagation formulas for a MLP (using gradient descent) and implementing them is a worthwhile exercise, because it demystifies machine learning. I totally agree that ML in practice is more about deciding what to optimize.
Paul Orland How is this book different from Think Stats?
I don’t know that book, but presumably the difference is that it covers stats while my book does not 🙂
Paul Orland I have absolutely and thoroughly enjoyed reading your book. Not only are the progressions throughout the examples highly engaging and approachable to a wide audience, but the accompanying explanations are very clever and entertaining. I’m curious to know if you’re planning on ever applying your teaching style and examples to the world of geospatial processing (2D and 3D) algorithms? I think you’d do a rather great job at making a topic such as that approachable to the rapidly growing community of practitioners that are entering that specific space.
Glad you liked it! I don’t know much about geospatial algorithms, but sounds like it could be a cool application for 2D and 3D geometry. Can you recommend any good references for me to get started?
Ben Wilson do you mean projection algorithms? or stuff like spatial indexes, spatial search and so on?
great circles intersection, polygonal collision, distance measurements with z-components, et. al. I think that with your presentation style you could write an effective primer to basically GIS ( 🙂 ) http://wiki.gis.com/wiki/index.php/New_to_GIS#Related_links
nice :)) I used to do a lot of spatial stuff (postgis mainly but also a bit of python). Would be nice to have a deeper dive into some of the maths definitely
I know great developers who want to get into machine learning, but they are scared of math. The moment they see a formula, their mind goes blank.
What would you recommend them to do? How can they overcome this fear?
I think the fear comes from school/university when teachers just force math down our throat without really explaining why we should learn it
this may help to manage fear
Alexey Grigorev same problem goes for me
It was the same for me, but if you take your time to break a complicated formula to parts and dig into each of them, then after a couple of iterations it won’t be that scary
I usually learn difficult topics by finding good explanations
Roman G I have found there is a delicate tradeoff between lowering inhibition and keeping ones mental faculties intact with the method you propose 🙂
This one? https://xkcd.com/323/
I like this question because it is a problem I still have – sometimes I find myself looking at a page in a math textbook and thinking to myself this looks hopelessly complex.
Alexey Grigorev yes exactly!
I would say, start by writing down the formulas you do know or drawing diagrams (related to the topic), without looking at a book or other resources. That should show you where the holes in your understanding are. Then go back to the book and try to fill these holes as needed. Over time, you should get some more confidence!
Makes sense, thank you!
I would say, start by writing down the formulas you do know or drawing diagrams (related to the topic), without looking at a book or other resources. That should show you where the holes in your understanding are. Then go back to the book and try to fill these holes as needed. Over time, you should get some more confidence!
Why knowing math is a good idea for programmers? Might it be useful for developers even if they don’t need it at work?
I would say yes, I believe that algorithms are basically about breaking mathematical constructs into simple step by step processes or approximations, at school too much emphasis is given to paper math, I think more emphasis should be put into computational thinking in all engineering degrees not only computer science.
A lot of ideas transfer directly from math to computer programming: logic, functions, sets, etc. I’m a strong believer that math skills help you think more rigorously about software (of any kind) that you build
Also I’m curious what you think about the way math is taught at schools and universities. Is there something you’d change?
Yes, I would change some things! I think math classrooms should be flipped, meaning record lectures so students can consume the content at their own pace, independently. Then save class time for exploration and problem solving. I would also say that most people who take math in college are not going to become mathematicians, so there should be a greater emphasis on applied math and using technology to solve math problems (e.g. Python or even Excel)
Nice! How would you motivate students to watch the content at their own pace before the classes?
That’s a problem I also have when teaching adults =)
Obviously some students are more self-motivated than others, but I think this would increase the quality of the learning experience and have most people more engaged overall.
Yes, makes sense! Thank you!