Questions and Answers
In my opinion, if your AI is not your primary business then AI on edge makes a lot of sense. You’re trading freedom of your model for security as the model maintainer.
Example: Your AI does style transfer for makeup. Then it’s amazing to run this on the browser on the client and get more people to click purchase for their makeup or even hair dye packages.
However, if your model is proprietary and your work is to apply the makeup/hairdye it’s harder to protect your model, as it must be downloaded to the client.
Got it, nice example Gant. So depending on the use case it can certainly make sense to have client- and server-based deployments in parallel, at least in the short- to medium-term.
absolutely. And you can easily convert TF to TFJS models. Most of them convert just fine.
That would have been my follow-up question 🙂
Thanks a lot for your response
Hello Gant, I know you from Merve Noyan’s podcast 🙂
A lot of people who aspring data scientist know python and libraries for this area. Machine Learning and Deep learning so important of course but I think python isn’t enough for ML Engineer or Data Scientist. This titles require Back-end of course. What do you think about back-end? there are 3 ways in back. You would deploy your model Web server, mobile application and of course Devops(docker,kubernetes, you can dockerize your model.)
My question is which one will populer in future ?
I want to add one more skill to myself but I don’t know which one I have to add. Actually this question is a little personal question because, You don’t know which one I will love or hate, but I wanted to ask.
If I understand correctly, you’re interested in my preferred backend solution for a web-hosted model.
I think microservices/cloud functions are perfect for models in the backend, because they are very “pay as you go” with infinite scalability.
So I would prefer setting up an AWS lambda that handles a result and reports back the results.
These are not limited by GPUs either. The whole problem of dynamically allocating resources for your server model is handled by the service, and faster and more scalable than either of us could hope.
The largest benefit, is that when the model is not in use, the costs are quite minimal. I’ve got several free AWS files/services I have released in the wild from past conference talks. They’ve never gone over $10 USD a month.
Okey, I got it. Thankss :heart:
With TF.js, how does a training pipeline look like? Do you do training in python and then export the model to use in JS, or you do the entire thing in JS?
In Python, there are many nice libraries for data augmentation and different image manipulations. I’m wondering what happens when we move to the browser. Do we need to re-implement all this functionality in JS?
So if you’re needing the power of python libs, I generally use Python to train for my advanced networks and then convert it down to JS.
In the book, I show you object localization and detection. Both of those I decided to do in Python for ease and then convert over.
But you can definitely do it in JS, and I show how.
In the book the Tic Tac Toe model was trained in JS, and in chapters 8, 9, 10, 11, 12 all models are trained in JS.
I think Python wins in having a history of lots of libs. But the speed of training in Node is no joke!
What are some good image manipulation libraries in node?
Similar to imgaug is this: https://github.com/piercus/image-augment
I think you’ll see more and more JS level tools coming in the next 2 years
Oh nice, thank you!
> seasoned AI veterans and web developers alike can help propel the future of AI-driven websites
Which type of folks do you find using
Tensorflow.js more - seasoned AI veterans, or web developers? (Am assuming that the intersection of these two camps may be somewhat smaller that the union 🙂 )
using more: web developers are playing with all kinds of amazing tech and will continue to grow. I think in quantity the current winner is web devs.
However, in quality, I think the best projects are coming from seasoned AI veterans who are leveraging AI in web. The most profitable and impressive projects are people who know how to make this new avenue of AI sing.
Let’s say I’m a python developer who wants to learn Tensorflow.js to use in my projects. How would you suggest to get started? Would your book will be enough or I already need to know some JS?
Is there a good learn-JS-in-15-minutes tutorial for those who already know how to program?
I think if you just read/tinker with some js you can pick it up in 15mins (just remember those curly brackets ;)) (and also async stuff can trip you up if you’re not careful)
Glancing through this, here’s a 1hr jumpstart that seems decent
Hello, Gant! I have tried porting some my TensorFlow2.0 models to TF.js and have been quite disappointed by the fact that lots of cool new features are poorly supported (for example, feature columns). Would you agree with my opinion that feature-wise TensorFlow.js is cursed to always lag behind TensorFlow? Or maybe one day it will take the lead?
Yes. TFJS will always lag behind. However, in a few ways it is ahead. TFJS 1.0 was using TF 2 syntax before TF 2 was released.
I think you’ll continue to see TFJS parity be a priority, but yes, lagging behind.
Thanks for your answer. I hope this project will get more attention thus voluntary contribution from the open source community.
Would you crazy kids be interested in seeing some internal graphics and process stuff?
That will be awesome for sure
I had to do my own graphics. I did OK
Some of my examples were just plain silly
The editors were AMAZING though, they loved everything I had
I’m quite proud of this diagram… but I know what they make will be better for print
Also, I used coworkers in examples. This will let me know if they actually read the book
I was worried if some of my graphics would work in B&W, but they gave them the thumbs up!!!!
Wow, looks interesting already
Thanks for your time with us here, waiting forbthe interanl graphics..
Here’s my question
does TensorFlow.js have access to the file system in the browser host environment?
if it does not, that means available data resources are limited and can put restrictions on file sizes needed for training and verifying, right?
TensorFlow.js can run on client or server.
So if you’re running your training on a machine using Node.js you have direct access to the filesystem via Node.js. Then you’re just managing stream and RAM.
If you’re running on the browser, you are sandboxed. This means you cannot access the filesystem and you’ll need to load your data via compatible URIs. Browsers have DBs in local storage, RAM, cookies, and all kinds of fun places you can gather data.
Because of the sandbox, I would recommend only minor amounts of training to happen in the browser if you have a significant data source. The browser is much better at utilizing trained models rather than training models.
This does break down when you want customization for your client, however. So if you’re going to do transfer learning or federated learning, you can get real benefits from training in the browser, but yes, it’s constrained.
My suggestion, do base training in Node.js, then ship the model. Do transfer/federated learning in the browser as needed.
Super, looks clearer now
Which chapter did you enjoy the most? And which one was the most challenging for you?
That’s a tough one!!!
I guess the capstone project was pretty hard for me. Bc I had to come up with something that tied all the lessons from the book together into a cohesive unit. It was a fantastic end for the book though! But tough!
Second hardest chapter is chapter 1… I rewrote that damn thing 10xs
My favorite chapter was probably Chapter 10, which was simple image training. It felt fun and rewarding.
Chapter 6 was a secondmost favorite, because we hook a model up to a webcam!
Sounds like a lot of fun!
Also, as an author myself, I’m really impressed by how fast you finished your book. Can you tell us a bit about your process, and how you organized your time to work on the book?
I have to say the pandemic helped. I was able to get into a writing frenzy. My editor told me, “Ride that wave!” and so I did. The more I wrote the easier it became and I hit a traction point where I realized I was running out of chapters. At that point, I had to assess what makes it in the book and what didn’t.
After that strange hiccup, the finish line was in sight and I used the rest of my frenzy to finish the book.
Since I finished the book, I’ve done a whole lot of nothing! I think I’m recovering. I don’t even read cool projects right now, I binge watched a whole season of a Netflix show.
How did get into this flow mode? Or it just happened itself?
To be honest, I didn’t get into it at first. I had so much trouble writing chapter 1. However, like gradient decent, each time I took a step that felt right, I leaned into it. Next thing I knew I was running.
One thing that reallllly helped, was having people review chapters. That helped me see large leaps of logic I made, and when I went back to fill in the missing gaps, my mindset carried me to how they should see the next thing and the next. Sort of like reading the book from an outside perspective, I had a conversation with a hypothetical audience.
Hi Gant can you compare the back-end features of tfjs-node against Python? What’s missing from tfjs-node that would encourage more use of JS on the back-end? Does tfjs-node have any advantages over Python on the back-end?
I feel like web frameworks that use Python are generally treated as second-class in the web world. Node for cloud etc is easy.
Node has even outperformed Python in quite a few benchmarks.
IMO, if you’re using backend, it needs to match the team maintaining it. Focus on dev preference.
The big advantage is that the code is portable to client, and IoT
Similar to portability, you can provide the same computations in the client, on customer hardware, and at scale on the server.
I’m just trying to understand why I don’t see more discussion of TFJS on the backend? I don’t have stats, but I see much more traffic about TF+Python than TFJS in Node. Like you said, I’d expect the web backend work to be more JS/Java, but I just don’t see much about that in the wild.
Wendy Mak’s comment about async/await makes me think that Data Scientists might just be more comfortable with Python’s less asynchronous nature?
Python was the spoken language for primary data culture, so it will take time for new solutions to root.
FYI, tfjs-node-gpu works just fine in node and uses GPUs when available using the main TF Cuda backend. Interesting question about async JS. Perhaps many folks feel more comfortable with the more traditional Python TF binding?
I’m find with async await. It’s common in JS, so it feels natural.
As for the second part, tfjs-node-gpu works great for accesing the GPU on node.
The browser uses WebGL, and they have an experiment with WebGPU which is promising.
Lastly, they are doing some amazing stuff in WASM with threads! I know this is a bit off-topic, but it’s cool.
> I’m find with async await. It’s common in JS, so it feels natural.
ah, cool, although, I think I like synchronus better lol – the async things gave me no end of headaches when I was a js dev 😂 (well, I was also trying to munge together something that takes about 10 steps each of which has a callback…)
Thanks for the GPU.js tip!
Oh, GPU.js is a different library. Brain.js uses GPU.js
WebGL and WebGPU are being supported as backends for TFJS
we need more acronyms ^ 😂
I know! I hadn’t heard there was a (limited) JS Cuda binding until you mentioned it.
Hi, Gant Thanks for participating book of the week.
I have a question about TF.js and federated learning. My question can be stupid, because I do not understand federated learning much. But if I understand correctly, then
- Federated learning proposes a model training on a distributed system without collecting the data in one place.
- 3rd-party cookie will be replaced with FLoC (federated learning of cohorts).
Because FL(oC) involves a distributed system, especially web clients/browsers for FLoC, it is natural to use TF.js to implement FL so that FLoC is one of the use cases. So my question is: Do you think TF.js can be mainstream of FL(oC)?
(Recently Google released a new version of chrome on which FLoC works, but I do not know its implementation.)
Federated learning is pretty complicated, and there’s the idea and implementation details.
The idea (at least in my head).
- Create a model
- Send copies to people
- Use transfer learning to personalize and improve the model
- Send back the improved models (NOT the user’s data)
- Improve the core model that is distributed.
I’m unaware of the FL(oC) specifics, but I significantly believe Google has a lot of benefit in getting federated learning going. I think you can expect TFJS to be a core contributor to this.
I know Google is seeking lots of adjustments to W3C standards based on TFJS, but it’s hard to know which pieces are prioritized and why.
I have no doubt that this will emerge. I think there’s a lot of green opportunities in TFJS for sure.