brief contentsPART 1 MOTIVATION AND BASIC CONCEPTS. .................................11 ■ Deep learning and JavaScript 3PART 2 A GENTLE INTRODUCTION TO TENSORFLOW.JS . ............. 352 ■ Getting started: Simple linear regression in TensorFlow.js 373 ■ Adding nonlinearity: Beyond weighted sums 794 ■ Recognizing images and sounds using convnets 1175 ■ Transfer learning: Reusing pretrained neural networks 152PART 3 ADVANCED DEEP LEARNING WITH TENSORFLOW.JS. ....... 1996 ■ Working with data 2017 ■ Visualizing data and models 2468 ■ Underfitting, overfitting, and the universal workflowof machine learning 2739 ■ Deep learning for sequences and text 29210 ■ Generative deep learning 33411 ■ Basics of deep reinforcement learning 371PART 4 SUMMARY AND CLOSING WORDS . .................................. 41512 ■ Testing, optimizing, and deploying models 41713 ■ Summary, conclusions, and beyond 453viicontentsforeword xiiipreface xvacknowledgments xviiabout this book xixabout the authors xxiiabout the cover illustration xxiiiPART 1 MOTIVATION AND BASIC CONCEPTS....................11 Deep learning and JavaScript 31.1 Artificial intelligence, machine learning, neural networks,and deep learning 6Artificial intelligence 6 ■ Machine learning: How it differs fromtraditional programming 7 ■ Neural networks and deeplearning 12 ■ Why deep learning? Why now? 161.2 Why combine JavaScript and machine learning? 18Deep learning with Node.js 24 ■ The JavaScript ecosystem 251.3 Why TensorFlow.js? 27A brief history of TensorFlow, Keras, and TensorFlow.js 27 ■ WhyTensorFlow.js: A brief comparison with similar libraries 31 ■ Howis TensorFlow.js being used by the world? 31 ■ What this book willand will not teach you about TensorFlow.js 32viii CONTENTSPART 2 A GENTLE INTRODUCTION TOTENSORFLOW.JS ..............................................352 Getting started: Simple linear regression in TensorFlow.js 372.1 Example 1: Predicting the duration of a download usingTensorFlow.js 38Project overview: Duration prediction 38 ■ A note on code listingsand console interactions 39 ■ Creating and formatting thedata 40 ■ Defining a simple model 43 ■ Fitting the modelto the training data 46 ■ Using our trained model to makepredictions 48 ■ Summary of our first example 492.2 Inside Model.fit(): Dissecting gradient descentfrom example 1 50The intuitions behind gradient-descent optimization 50Backpropagation: Inside gradient descent 562.3 Linear regression with multiple input features 59The Boston Housing Prices dataset 60 ■ Getting and running theBoston-housing project from GitHub 61 ■ Accessing the Bostonhousing data 63 ■ Precisely defining the Boston-housingproblem 65 ■ A slight diversion into data normalization 66Linear regression on the Boston-housing data 702.4 How to interpret your model 74Extracting meaning from learned weights 74 ■ Extracting internalweights from the model 75 ■ Caveats on interpretability 773 Adding nonlinearity: Beyond weighted sums 793.1 Nonlinearity: What it is and what it is good for 80Building the intuition for nonlinearity in neural networks 82Hyperparameters and hyperparameter optimization 893.2 Nonlinearity at output: Models for classification 92What is binary classification? 92 ■ Measuring the quality ofbinary classifiers: Precision, recall, accuracy, and ROC curves 96The ROC curve: Showing trade-offs in binary classification 99Binary cross entropy: The loss function for binary classification 1033.3 Multiclass classification 106One-hot encoding of categorical data 107 ■ Softmaxactivation 109 ■ Categorical cross entropy: The loss functionfor multiclass classification 111 ■ Confusion matrix: Fine-grainedanalysis of multiclass classification 113CONTENTS ix4 Recognizing images and sounds using convnets 1174.1 From vectors to tensors: Representing images 118The MNIST dataset 1194.2 Your first convnet 120conv2d layer 122 ■ maxPooling2d layer 126 ■ Repeatingmotifs of convolution and pooling 127 ■ Flatten and denselayers 128 ■ Training the convnet 130 ■ Using a convnet tomake predictions 1344.3 Beyond browsers: Training models faster usingNode.js 137Dependencies and imports for using tfjs-node 137 ■ Saving themodel from Node.js and loading it in the browser 1424.4 Spoken-word recognition: Applying convnets onaudio data 144Spectrograms: Representing sounds as images 1455 Transfer learning: Reusing pretrained neural networks 1525.1 Introduction to transfer learning: Reusing pretrainedmodels 153Transfer learning based on compatible output shapes: Freezinglayers 155 ■ Transfer learning on incompatible output shapes:Creating a new model using outputs from the base model 161Getting the most out of transfer learning through fine-tuning: Anaudio example 1745.2 Object detection through transfer learning on aconvnet 185A simple object-detection problem based on synthesized scenes 186Deep dive into simple object detection 187PART 3 ADVANCED DEEP LEARNING WITHTENSORFLOW.JS ............................................1996 Working with data 2016.1 Using tf.data to manage data 202The tf.data.Dataset object 203 ■ Creating a tf.data.Dataset 203Accessing the data in your dataset 209 ■ Manipulating tfjs-datadatasets 2106.2 Training models with model.fitDataset 214x CONTENTS6.3 Common patterns for accessing data 220Working with CSV format data 220 ■ Accessing video data usingtf.data.webcam() 225 ■ Accessing audio data usingtf.data.microphone() 2286.4 Your data is likely flawed: Dealing with problemsin your data 230Theory of data 231 ■ Detecting and cleaning problems withdata 2356.5 Data augmentation 2427 Visualizing data and models 2467.1 Data visualization 247Visualizing data using tfjs-vis 247 ■ An integrative case study:Visualizing weather data with tfjs-vis 2557.2 Visualizing models after training 260Visualizing the internal activations of a convnet 262Visualizing what convolutional layers are sensitive to: Maximallyactivating images 265 ■ Visual interpretation of a convnet’sclassification result 2698 Underfitting, overfitting, and the universal workflowof machine learning 2738.1 Formulation of the temperature-prediction problem 2748.2 Underfitting, overfitting, and countermeasures 278Underfitting 278 ■ Overfitting 280 ■ Reducing overfittingwith weight regularization and visualizing it working 2828.3 The universal workflow of machine learning 2879 Deep learning for sequences and text 2929.1 Second attempt at weather prediction:Introducing RNNs 294Why dense layers fail to model sequential order 294 ■ How RNNsmodel sequential order 2969.2 Building deep-learning models for text 305How text is represented in machine learning: One-hot and multi-hotencoding 306 ■ First attempt at the sentiment-analysisproblem 308 ■ A more efficient representation of text: Wordembeddings 310 ■ 1D convnets 312CONTENTS xi9.3 Sequence-to-sequence tasks with attentionmechanism 321Formulation of the sequence-to-sequence task 321 ■ The encoderdecoder architecture and the attention mechanism 324 ■ Deep diveinto the attention-based encoder-decoder model 32710 Generative deep learning 33410.1 Generating text with LSTM 335Next-character predictor: A simple way to generate text 335The LSTM-text-generation example 337 ■ Temperature:Adjustable randomness in the generated text 34210.2 Variational autoencoders: Finding an efficient andstructured vector representation of images 345Classical autoencoder and VAE: Basic ideas 345 ■ A detailedexample of VAE: The Fashion-MNIST example 34910.3 Image generation with GANs 356The basic idea behind GANs 357 ■ The building blocks ofACGAN 360 ■ Diving deeper into the training of ACGAN 363Seeing the MNIST ACGAN training and generation 36611 Basics of deep reinforcement learning 37111.1 The formulation of reinforcement-learningproblems 37311.2 Policy networks and policy gradients: The cart-poleexample 376Cart-pole as a reinforcement-learning problem 376 ■ Policynetwork 378 ■ Training the policy network: The REINFORCEalgorithm 38111.3 Value networks and Q-learning: The snake gameexample 389Snake as a reinforcement-learning problem 389 ■ Markov decisionprocess and Q-values 392 ■ Deep Q-network 396 ■ Trainingthe deep Q-network 399PART 4 SUMMARY AND CLOSING WORDS .....................41512 Testing, optimizing, and deploying models 41712.1 Testing TensorFlow.js models 418Traditional unit testing 419 ■ Testing with golden values 422Considerations around continuous training 424xii CONTENTS12.2 Model optimization 425Model-size optimization through post-training weightquantization 426 ■ Inference-speed optimization usingGraphModel conversion 43412.3 Deploying TensorFlow.js models on various platformsand environments 439Additional considerations when deploying to the web 439Deployment to cloud serving 440 ■ Deploying to a browserextension, like Chrome Extension 441 ■ Deploying TensorFlow.jsmodels in JavaScript-based mobile applications 443 ■ DeployingTensorFlow.js models in JavaScript-based cross-platform desktopapplications 445 ■ Deploying TensorFlow.js models on WeChatand other JavaScript-based mobile app plugin systems 447Deploying TensorFlow.js models on single-board computers 448Summary of deployments 45013 Summary, conclusions, and beyond 45313.1 Key concepts in review 454Various approaches to AI 454 ■ What makes deep learning standout among the subfields of machine learning 455 ■ How to thinkabout deep learning at a high level 455 ■ Key enablingtechnologies of deep learning 456 ■ Applications andopportunities unlocked by deep learning in JavaScript 45713.2 Quick overview of the deep-learning workflowand algorithms in TensorFlow.js 458The universal workflow of supervised deep learning 458Reviewing model and layer types in TensorFlow.js: A quickreference 460 ■ Using pretrained models from TensorFlow.js 465The space of possibilities 468 ■ Limitations of deep learning 47013.3 Trends in deep learning 47313.4 Pointers for further exploration 474Practice real-world machine-learning problems on Kaggle 474Read about the latest developments on arXiv 475 ■ Explore theTensorFlow.js Ecosystem 475 appendix A Installing tfjs-node-gpu and its dependencies 477appendix B A quick tutorial of tensors and operations in TensorFlow.js 482glossary 507index 519
评论