java核心基础总结.pdf

JAVA核心知识点整理.pdf

1. 目录1. 目录.........................................................................................................................................................12. JVM.......................................................................................................................................................192.1. 线程 ......................................................................................................................................................202.2. JVM 内存区域 .....................................................................................................................................212.2.1. 程序计数器(线程私有)................................................................................................................222.2.2. 虚拟机栈(线程私有)....................................................................................................................222.2.3. 本地方法区(线程私有)................................................................................................................232.2.4. 堆（Heap-线程共享）-运行时数据区 ......................................................................................232.2.5. 方法区/永久代（线程共享） .....................................................................................................232.3. JVM 运行时内存 .................................................................................................................................242.3.1. 新生代 ..........................................................................................................................................242.3.1.1. Eden 区....................................................................................................................................................242.3.1.2. ServivorFrom...........................................................................................................................................242.3.1.3. ServivorTo ..............................................................................................................................................242.3.1.4. MinorGC 的过程（复制->清空->互换） .......................................................................................241：eden、servicorFrom 复制到 ServicorTo，年龄 1...................................................................................252：清空 eden、servicorFrom.............................................................................................................................253：ServicorTo 和 ServicorFrom 互换................................................................................................................252.3.2. 老年代 ..........................................................................................................................................252.3.3. 永久代 ..........................................................................................................................................252.3.3.1. JAVA8 与元数据.................................................................................................................................252.4. 垃圾回收与算法 ..................................................................................................................................262.4.1. 如何确定垃圾 ..............................................................................................................................262.4.1.1. 引用计数法...............................................................................................................................................262.4.1.2. 可达性分析...............................................................................................................................................262.4.2. 标记清除算法（Mark-Sweep） ..............................................................................................272.4.3. 复制算法（copying）.................................................................................................................272.4.4. 标记整理算法(Mark-Compact)..................................................................................................282.4.5. 分代收集算法 ..............................................................................................................................292.4.5.1. 新生代与复制算法 ..............................................................................................................................292.4.5.2. 老年代与标记复制算法 ......................................................................................................................292.5. JAVA 四中引用类型 ...........................................................................................................................302.5.1. 强引用 ..........................................................................................................................................302.5.2. 软引用 ..........................................................................................................................................302.5.3. 弱引用 ..........................................................................................................................................302.5.4. 虚引用 ..........................................................................................................................................302.6. GC 分代收集算法 VS 分区收集算法................................................................................................302.6.1. 分代收集算法 ..............................................................................................................................302.6.1.1. 在新生代-复制算法.............................................................................................................................302.6.1.2. 在老年代-标记整理算法.....................................................................................................................302.6.2. 分区收集算法 ..............................................................................................................................312.7. GC 垃圾收集器 ...................................................................................................................................312.7.1. Serial 垃圾收集器（单线程、复制算法）................................................................................312.7.2. ParNew 垃圾收集器（Serial 多线程）...................................................................................312.7.3. Parallel Scavenge 收集器（多线程复制算法、高效）..........................................................322.7.4. Serial Old 收集器（单线程标记整理算法 ） ...........................................................................322.7.5. Parallel Old 收集器（多线程标记整理算法） .........................................................................332.7.6. CMS 收集器（多线程标记清除算法）.....................................................................................332.7.6.1. 初始标记..............................................................................................................................................3313/04/2018 Page 2 of 2832.7.6.2. 并发标记..............................................................................................................................................342.7.6.3. 重新标记..............................................................................................................................................342.7.6.4. 并发清除..............................................................................................................................................342.7.7. G1 收集器....................................................................................................................................342.8. JAVA IO/NIO.......................................................................................................................................342.8.1. 阻塞 IO 模型 ................................................................................................................................342.8.2. 非阻塞 IO 模型 ............................................................................................................................352.8.3. 多路复用 IO 模型 ........................................................................................................................352.8.4. 信号驱动 IO 模型 ........................................................................................................................362.8.5. 异步 IO 模型 ................................................................................................................................362.8.1. JAVA IO 包..................................................................................................................................362.8.2. JAVA NIO....................................................................................................................................372.8.2.1. NIO 的缓冲区 .....................................................................................................................................382.8.2.2. NIO 的非阻塞 .....................................................................................................................................382.8.3. Channel .......................................................................................................................................402.8.4. Buffer............................................................................................................................................402.8.5. Selector........................................................................................................................................402.9. JVM 类加载机制 .................................................................................................................................412.9.1.1. 加载 ..........................................................................................................................................................412.9.1.2. 验证 ..........................................................................................................................................................412.9.1.3. 准备 ..........................................................................................................................................................412.9.1.4. 解析 ..........................................................................................................................................................412.9.1.5. 符号引用..............................................................................................................................................422.9.1.6. 直接引用..............................................................................................................................................422.9.1.7. 初始化 ......................................................................................................................................................422.9.1.8. 类构造器<client>..............................................................................................................................422.9.2. 类加载器 ......................................................................................................................................422.9.2.1. 启动类加载器(Bootstrap ClassLoader) .........................................................................................432.9.2.2. 扩展类加载器(Extension ClassLoader)..........................................................................................432.9.2.3. 应用程序类加载器(Application ClassLoader)： ..........................................................................432.9.3. 双亲委派 ......................................................................................................................................432.9.4. OSGI（动态模型系统） ............................................................................................................442.9.4.1. 动态改变构造......................................................................................................................................442.9.4.2. 模块化编程与热插拔 ..........................................................................................................................443. JAVA 集合............................................................................................................................................453.1. 接口继承关系和实现 ..........................................................................................................................453.2. LIST.......................................................................................................................................................473.2.1. ArrayList（数组）.......................................................................................................................473.2.2. Vector（数组实现、线程同步）...............................................................................................473.2.3. LinkList（链表）.........................................................................................................................473.3. SET .......................................................................................................................................................483.3.1.1. HashSet（Hash 表）.............................................................................................................................483.3.1.2. TreeSet（二叉树） ................................................................................................................................493.3.1.3. LinkHashSet（HashSet LinkedHashMap）...................................................................................493.4. MAP.......................................................................................................................................................503.4.1. HashMap（数组 链表 红黑树）.............................................................................................503.4.1.1. JAVA7 实现 .............................................................................................................................................503.4.1.2. JAVA8 实现 .............................................................................................................................................513.4.2. ConcurrentHashMap..................................................................................................................513.4.2.1. Segment 段..............................................................................................................................................513.4.2.2. 线程安全（Segment 继承 ReentrantLock 加锁）..............................................................................513.4.2.3. 并行度（默认 16） .................................................................................................................................523.4.2.4. Java8 实现 （引入了红黑树） ..............................................................................................................5213/04/2018 Page 3 of 2833.4.3. HashTable（线程安全） ...........................................................................................................533.4.4. TreeMap（可排序） ..................................................................................................................533.4.5. LinkHashMap（记录插入顺序）..............................................................................................534. JAVA 多线程并发.................................................................................................................................544.1.1. JAVA 并发知识库 .......................................................................................................................544.1.2. JAVA 线程实现/创建方式 ..........................................................................................................544.1.2.1. 继承 Thread 类........................................................................................................................................544.1.2.2. 实现 Runnable 接口。............................................................................................................................544.1.2.3. ExecutorService、Callable<Class>、Future 有返回值线程.............................................................554.1.2.4. 基于线程池的方式...................................................................................................................................564.1.3. 4 种线程池 ...................................................................................................................................564.1.3.1. newCachedThreadPool.........................................................................................................................574.1.3.2. newFixedThreadPool.............................................................................................................................574.1.3.3. newScheduledThreadPool....................................................................................................................584.1.3.4. newSingleThreadExecutor .................................................................................................................584.1.4. 线程生命周期(状态)....................................................................................................................584.1.4.1. 新建状态（NEW） .................................................................................................................................584.1.4.2. 就绪状态（RUNNABLE）：.................................................................................................................594.1.4.3. 运行状态（RUNNING）： ....................................................................................................................594.1.4.4. 阻塞状态（BLOCKED）：....................................................................................................................59等待阻塞（o.wait->等待对列）：......................................................................................................................59同步阻塞(lock->锁池) ..........................................................................................................................................59其他阻塞(sleep/join) ............................................................................................................................................594.1.4.5. 线程死亡（DEAD）................................................................................................................................59正常结束................................................................................................................................................................59异常结束................................................................................................................................................................59调用 stop...............................................................................................................................................................594.1.5. 终止线程 4 种方式 ......................................................................................................................604.1.5.1. 正常运行结束...........................................................................................................................................604.1.5.2. 使用退出标志退出线程...........................................................................................................................604.1.5.3. Interrupt 方法结束线程...........................................................................................................................604.1.5.4. stop 方法终止线程（线程不安全）.......................................................................................................614.1.6. sleep 与 wait 区别.......................................................................................................................614.1.7. start 与 run 区别..........................................................................................................................624.1.8. JAVA 后台线程 ...........................................................................................................................624.1.9. JAVA 锁 .......................................................................................................................................634.1.9.1. 乐观锁 ......................................................................................................................................................634.1.9.2. 悲观锁 ......................................................................................................................................................634.1.9.3. 自旋锁 ......................................................................................................................................................63自旋锁的优缺点....................................................................................................................................................63自旋锁时间阈值（1.6 引入了适应性自旋锁）..................................................................................................63自旋锁的开启........................................................................................................................................................644.1.9.4. Synchronized 同步锁..............................................................................................................................64Synchronized 作用范围.......................................................................................................................................64Synchronized 核心组件.......................................................................................................................................64Synchronized 实现...............................................................................................................................................644.1.9.5. ReentrantLock.........................................................................................................................................66Lock 接口的主要方法...........................................................................................................................................66非公平锁................................................................................................................................................................66公平锁....................................................................................................................................................................67ReentrantLock 与 synchronized ........................................................................................................................67ReentrantLock 实现.............................................................................................................................................67Condition 类和 Object 类锁方法区别区别.........................................................................................................68tryLock 和 lock 和 lockInterruptibly 的区别........................................................................................................684.1.9.6. Semaphore 信号量.................................................................................................................................68实现互斥锁（计数器为 1）.................................................................................................................................68代码实现................................................................................................................................................................68Semaphore 与 ReentrantLock ...........................................................................................................................694.1.9.7. AtomicInteger..........................................................................................................................................6913/04/2018 Page 4 of 2834.1.9.8. 可重入锁（递归锁）...............................................................................................................................694.1.9.9. 公平锁与非公平锁...................................................................................................................................70公平锁（Fair）.....................................................................................................................................................70非公平锁（Nonfair）...........................................................................................................................................704.1.9.10. ReadWriteLock 读写锁......................................................................................................................70读锁........................................................................................................................................................................70写锁........................................................................................................................................................................704.1.9.11. 共享锁和独占锁 ..................................................................................................................................70独占锁....................................................................................................................................................................70共享锁....................................................................................................................................................................704.1.9.12. 重量级锁（Mutex Lock） ................................................................................................................714.1.9.13. 轻量级锁..............................................................................................................................................71锁升级....................................................................................................................................................................714.1.9.14. 偏向锁..................................................................................................................................................714.1.9.15. 分段锁..................................................................................................................................................714.1.9.16. 锁优化..................................................................................................................................................71减少锁持有时间....................................................................................................................................................72减小锁粒度............................................................................................................................................................72锁分离....................................................................................................................................................................72锁粗化....................................................................................................................................................................72锁消除....................................................................................................................................................................724.1.10. 线程基本方法 ..............................................................................................................................724.1.10.1. 线程等待（wait）...............................................................................................................................734.1.10.2. 线程睡眠（sleep）.............................................................................................................................734.1.10.3. 线程让步（yield）..............................................................................................................................734.1.10.4. 线程中断（interrupt）........................................................................................................................734.1.10.5. Join 等待其他线程终止......................................................................................................................744.1.10.6. 为什么要用 join()方法？ ....................................................................................................................744.1.10.7. 线程唤醒（notify）.............................................................................................................................744.1.10.8. 其他方法：..........................................................................................................................................744.1.11. 线程上下文切换 ..........................................................................................................................754.1.11.1. 进程......................................................................................................................................................754.1.11.2. 上下文..................................................................................................................................................754.1.11.3. 寄存器..................................................................................................................................................754.1.11.4. 程序计数器..........................................................................................................................................754.1.11.5. PCB-“切换桢”.................................................................................................................................754.1.11.6. 上下文切换的活动： ..........................................................................................................................764.1.11.7. 引起线程上下文切换的原因 ..............................................................................................................764.1.12. 同步锁与死锁 ..............................................................................................................................764.1.12.1. 同步锁..................................................................................................................................................764.1.12.2. 死锁......................................................................................................................................................764.1.13. 线程池原理 ..................................................................................................................................764.1.13.1. 线程复用..............................................................................................................................................764.1.13.2. 线程池的组成......................................................................................................................................764.1.13.3. 拒绝策略..............................................................................................................................................784.1.13.4. Java 线程池工作过程 .........................................................................................................................784.1.14. JAVA 阻塞队列原理....................................................................................................................794.1.14.1. 阻塞队列的主要方法 ..........................................................................................................................80插入操作：............................................................................................................................................................80获取数据操作：....................................................................................................................................................814.1.14.2. Java 中的阻塞队列 .............................................................................................................................814.1.14.3. ArrayBlockingQueue（公平、非公平） .......................................................................................824.1.14.4. LinkedBlockingQueue（两个独立锁提高并发） .........................................................................824.1.14.5. PriorityBlockingQueue（compareTo 排序实现优先）..............................................................824.1.14.6. DelayQueue（缓存失效、定时任务 ）..........................................................................................824.1.14.7. SynchronousQueue（不存储数据、可用于传递数据）..............................................................834.1.14.8. LinkedTransferQueue......................................................................................................................8313/04/2018 Page 5 of 2834.1.14.9. LinkedBlockingDeque.....................................................................................................................834.1.15. CyclicBarrier、CountDownLatch、Semaphore 的用法........................................................844.1.15.1. CountDownLatch（线程计数器 ） ................................................................................................844.1.15.2. CyclicBarrier（回环栅栏-等待至 barrier 状态再全部同时执行） ...............................................844.1.15.3. Semaphore（信号量-控制同时访问的线程个数） .......................................................................854.1.16. volatile 关键字的作用（变量可见性、禁止重排序） .............................................................87变量可见性............................................................................................................................................................87禁止重排序............................................................................................................................................................87比 sychronized 更轻量级的同步锁.....................................................................................................................87适用场景................................................................................................................................................................874.1.17. 如何在两个线程之间共享数据...................................................................................................88将数据抽象成一个类，并将数据的操作作为这个类的方法.............................................................................88Runnable 对象作为一个类的内部类 ..................................................................................................................894.1.18. ThreadLocal 作用（线程本地存储）........................................................................................90ThreadLocalMap（线程的一个属性） ..............................................................................................................90使用场景................................................................................................................................................................914.1.19. synchronized 和 ReentrantLock 的区别 ..................................................................................914.1.19.1. 两者的共同点： ..................................................................................................................................914.1.19.2. 两者的不同点： ..................................................................................................................................924.1.20. ConcurrentHashMap 并发.........................................................................................................924.1.20.1. 减小锁粒度..........................................................................................................................................924.1.20.2. ConcurrentHashMap 分段锁..........................................................................................................92ConcurrentHashMap 是由 Segment 数组结构和 HashEntry 数组结构组成.................................................934.1.21. Java 中用到的线程调度 .............................................................................................................934.1.21.1. 抢占式调度：......................................................................................................................................934.1.21.2. 协同式调度：......................................................................................................................................934.1.21.3. JVM 的线程调度实现（抢占式调度） .............................................................................................944.1.21.4. 线程让出 cpu 的情况： .....................................................................................................................944.1.22. 进程调度算法 ..............................................................................................................................944.1.22.1. 优先调度算法......................................................................................................................................944.1.22.2. 高优先权优先调度算法 ......................................................................................................................954.1.22.3. 基于时间片的轮转调度算法 ..............................................................................................................964.1.23. 什么是 CAS（比较并交换-乐观锁机制-锁自旋） ..................................................................964.1.23.1. 概念及特性..........................................................................................................................................964.1.23.2. 原子包 java.util.concurrent.atomic（锁自旋） ...........................................................................974.1.23.3. ABA 问题.............................................................................................................................................984.1.24. 什么是 AQS（抽象的队列同步器）.........................................................................................98Exclusive 独占资源-ReentrantLock ...................................................................................................................99Share 共享资源-Semaphore/CountDownLatch ...............................................................................................99同步器的实现是 ABS 核心（state 资源状态计数）.......................................................................................100ReentrantReadWriteLock 实现独占和共享两种方式.....................................................................................1005. JAVA 基础..........................................................................................................................................1015.1.1. JAVA 异常分类及处理..............................................................................................................1015.1.1.1. 概念....................................................................................................................................................1015.1.1.2. 异常分类............................................................................................................................................101Error.....................................................................................................................................................................101Exception（RuntimeException、CheckedException） ...........................................................................1015.1.1.3. 异常的处理方式 ................................................................................................................................102遇到问题不进行具体处理，而是继续抛给调用者 （throw,throws）.........................................................102try catch 捕获异常针对性处理方式..................................................................................................................1025.1.1.4. Throw 和 throws 的区别： ............................................................................................................10213/04/2018 Page 6 of 283位置不同..............................................................................................................................................................102功能不同：..........................................................................................................................................................1025.1.2. JAVA 反射 .................................................................................................................................1035.1.2.1. 动态语言............................................................................................................................................1035.1.2.2. 反射机制概念 （运行状态中知道类所有的属性和方法）............................................................1035.1.2.3. 反射的应用场合 ................................................................................................................................103编译时类型和运行时类型..................................................................................................................................103的编译时类型无法获取具体方法......................................................................................................................1045.1.2.4. Java 反射 API....................................................................................................................................104反射 API 用来生成 JVM 中的类、接口或则对象的信息。 ............................................................................1045.1.2.5. 反射使用步骤（获取 Class 对象、调用对象方法） .....................................................................1045.1.2.6. 获取 Class 对象的 3 种方法 ............................................................................................................104调用某个对象的 getClass()方法.......................................................................................................................104调用某个类的 class 属性来获取该类对应的 Class 对象................................................................................104使用 Class 类中的 forName()静态方法(最安全/性能最好)............................................................................1045.1.2.7. 创建对象的两种方法 ........................................................................................................................105Class 对象的 newInstance().............................................................................................................................105调用 Constructor 对象的 newInstance()..........................................................................................................1055.1.3. JAVA 注解 .................................................................................................................................1065.1.3.1. 概念 ........................................................................................................................................................1065.1.3.2. 4 种标准元注解......................................................................................................................................106@Target 修饰的对象范围 .................................................................................................................................106@Retention 定义 被保留的时间长短...............................................................................................................106@Documented 描述-javadoc................................................................................................................................106@Inherited 阐述了某个被标注的类型是被继承的 ..............................................................................................1065.1.3.3. 注解处理器.............................................................................................................................................1075.1.4. JAVA 内部类 .............................................................................................................................1095.1.4.1. 静态内部类.............................................................................................................................................1095.1.4.2. 成员内部类.............................................................................................................................................1105.1.4.3. 局部内部类（定义在方法中的类）.....................................................................................................1105.1.4.4. 匿名内部类（要继承一个父类或者实现一个接口、直接使用 new 来生成一个对象的引用）.....1115.1.5. JAVA 泛型 .................................................................................................................................1125.1.5.1. 泛型方法（<E>）.............................................................................................................................1125.1.5.2. 泛型类<T> .............................................................................................................................................1125.1.5.3. 类型通配符? ..........................................................................................................................................1135.1.5.4. 类型擦除 ................................................................................................................................................1135.1.6. JAVA 序列化(创建可复用的 Java 对象).................................................................................113保存(持久化)对象及其状态到内存或者磁盘....................................................................................................113序列化对象以字节数组保持-静态成员不保存.................................................................................................113序列化用户远程对象传输..................................................................................................................................113Serializable 实现序列化 ....................................................................................................................................113ObjectOutputStream 和 ObjectInputStream 对对象进行序列化及反序列化...............................................113writeObject 和 readObject 自定义序列化策略................................................................................................113序列化 ID.............................................................................................................................................................113序列化并不保存静态变量..................................................................................................................................114序列化子父类说明..............................................................................................................................................114Transient 关键字阻止该变量被序列化到文件中............................................................................................1145.1.7. JAVA 复制 .................................................................................................................................1145.1.7.1. 直接赋值复制.........................................................................................................................................1145.1.7.2. 浅复制（复制引用但不复制引用的对象）.....................................................................................1145.1.7.3. 深复制（复制对象和其应用对象）.................................................................................................1155.1.7.4. 序列化（深 clone 一中实现）........................................................................................................1156. SPRING 原理 .....................................................................................................................................1166.1.1. Spring 特点................................................................................................................................1166.1.1.1. 轻量级................................................................................................................................................11613/04/2018 Page 7 of 2836.1.1.2. 控制反转............................................................................................................................................1166.1.1.3. 面向切面............................................................................................................................................1166.1.1.4. 容器....................................................................................................................................................1166.1.1.5. 框架集合............................................................................................................................................1166.1.2. Spring 核心组件........................................................................................................................1176.1.3. Spring 常用模块........................................................................................................................1176.1.4. Spring 主要包............................................................................................................................1186.1.5. Spring 常用注解........................................................................................................................1186.1.6. Spring 第三方结合....................................................................................................................1196.1.7. Spring IOC 原理........................................................................................................................1206.1.7.1. 概念 ........................................................................................................................................................1206.1.7.2. Spring 容器高层视图 ............................................................................................................................1206.1.7.3. IOC 容器实现.........................................................................................................................................120BeanFactory-框架基础设施..............................................................................................................................1201.1..1.1.1 BeanDefinitionRegistry 注册表.................................................................................................1211.1..1.1.2 BeanFactory 顶层接口..............................................................................................................1211.1..1.1.3 ListableBeanFactory .................................................................................................................1211.1..1.1.4 HierarchicalBeanFactory 父子级联..........................................................................................1211.1..1.1.5 ConfigurableBeanFactory.........................................................................................................1211.1..1.1.6 AutowireCapableBeanFactory 自动装配 ................................................................................1221.1..1.1.7 SingletonBeanRegistry 运行期间注册单例 Bean...................................................................1221.1..1.1.8 依赖日志框框.............................................................................................................................122ApplicationContext 面向开发应用....................................................................................................................122WebApplication 体系架构 .................................................................................................................................1236.1.7.4. Spring Bean 作用域..............................................................................................................................123singleton：单例模式（多线程下不安全） ......................................................................................................123prototype:原型模式每次使用时创建 ................................................................................................................124Request：一次 request 一个实例....................................................................................................................124session................................................................................................................................................................124global Session....................................................................................................................................................1246.1.7.5. Spring Bean 生命周期..........................................................................................................................124实例化..................................................................................................................................................................124IOC 依赖注入......................................................................................................................................................124setBeanName 实现............................................................................................................................................124BeanFactoryAware 实现...................................................................................................................................124ApplicationContextAware 实现.........................................................................................................................125postProcessBeforeInitialization 接口实现-初始化预处理..........................................................................125init-method..........................................................................................................................................................125postProcessAfterInitialization...........................................................................................................................125Destroy 过期自动清理阶段 ...............................................................................................................................125destroy-method 自配置清理 .............................................................................................................................1256.1.7.6. Spring 依赖注入四种方式....................................................................................................................126构造器注入..........................................................................................................................................................126setter 方法注入...................................................................................................................................................127静态工厂注入......................................................................................................................................................127实例工厂..............................................................................................................................................................1276.1.7.7. 5 种不同方式的自动装配......................................................................................................................1286.1.8. Spring APO 原理 ......................................................................................................................1296.1.8.1. 概念 ........................................................................................................................................................1296.1.8.2. AOP 核心概念 .......................................................................................................................................1296.1.8.1. AOP 两种代理方式 ...............................................................................................................................130JDK 动态接口代理 .............................................................................................................................................130CGLib 动态代理..................................................................................................................................................1316.1.8.2. 实现原理 ................................................................................................................................................1316.1.9. Spring MVC 原理......................................................................................................................1326.1.9.1. MVC 流程...............................................................................................................................................132Http 请求到 DispatcherServlet .......................................................................................................................133HandlerMapping 寻找处理器............................................................................................................................133调用处理器 Controller........................................................................................................................................13313/04/2018 Page 8 of 283Controller 调用业务逻辑处理后，返回 ModelAndView.................................................................................133DispatcherServlet 查询 ModelAndView ..........................................................................................................133ModelAndView 反馈浏览器 HTTP ...................................................................................................................1336.1.9.1. MVC 常用注解.......................................................................................................................................1336.1.10. Spring Boot 原理.......................................................................................................................1341. 创建独立的 Spring 应用程序.............................................................................................................................1342. 嵌入的 Tomcat，无需部署 WAR 文件.............................................................................................................1343. 简化 Maven 配置................................................................................................................................................1344. 自动配置 Spring .................................................................................................................................................1345. 提供生产就绪型功能，如指标，健康检查和外部配置...................................................................................1346. 绝对没有代码生成和对 XML 没有要求配置 [1]...............................................................................................1346.1.11. JPA 原理....................................................................................................................................1346.1.11.1. 事务....................................................................................................................................................1346.1.11.2. 本地事务............................................................................................................................................1346.1.11.1. 分布式事务........................................................................................................................................1356.1.11.1. 两阶段提交........................................................................................................................................1361 准备阶段...........................................................................................................................................................1362 提交阶段：.......................................................................................................................................................1366.1.12. Mybatis 缓存..............................................................................................................................1376.1.12.1. Mybatis 的一级缓存原理（sqlsession 级别）..............................................................................1386.1.12.2. 二级缓存原理（mapper 基本）.....................................................................................................138具体使用需要配置：..........................................................................................................................................1396.1.13. Tomcat 架构..............................................................................................................................1397. 微服务 .................................................................................................................................................1407.1.1. 服务注册发现 ............................................................................................................................1407.1.1.1. 客户端注册（zookeeper）..................................................................................................................1407.1.1.2. 第三方注册（独立的服务 Registrar）...............................................................................................1407.1.1.3. 客户端发现.............................................................................................................................................1417.1.1.4. 服务端发现.............................................................................................................................................1427.1.1.5. Consul....................................................................................................................................................1427.1.1.6. Eureka....................................................................................................................................................1427.1.1.7. SmartStack............................................................................................................................................1427.1.1.8. Etcd ........................................................................................................................................................1427.1.2. API 网关.....................................................................................................................................1427.1.2.1. 请求转发 ................................................................................................................................................1437.1.2.2. 响应合并 ................................................................................................................................................1437.1.2.3. 协议转换 ................................................................................................................................................1437.1.2.4. 数据转换 ................................................................................................................................................1437.1.2.5. 安全认证 ................................................................................................................................................1447.1.3. 配置中心 ....................................................................................................................................1447.1.3.1. zookeeper 配置中心.............................................................................................................................1447.1.3.2. 配置中心数据分类.................................................................................................................................1447.1.4. 事件调度（kafka）...................................................................................................................1447.1.5. 服务跟踪（starter-sleuth）...................................................................................................1447.1.6. 服务熔断（Hystrix） ................................................................................................................1457.1.6.1. Hystrix 断路器机制................................................................................................................................1467.1.7. API 管理.....................................................................................................................................1468. NETTY 与 RPC ..................................................................................................................................1478.1.1. Netty 原理..................................................................................................................................1478.1.2. Netty 高性能..............................................................................................................................1478.1.2.1. 多路复用通讯方式 ............................................................................................................................1478.1.2.1. 异步通讯 NIO....................................................................................................................................1488.1.2.2. 零拷贝（DIRECT BUFFERS 使用堆外直接内存）..........................................................................1498.1.2.3. 内存池（基于内存池的缓冲区重用机制） .........................................................................................1498.1.2.4. 高效的 Reactor 线程模型.....................................................................................................................149Reactor 单线程模型...........................................................................................................................................149Reactor 多线程模型...........................................................................................................................................15013/04/2018 Page 9 of 283主从 Reactor 多线程模型..................................................................................................................................1508.1.2.5. 无锁设计、线程绑定.............................................................................................................................1518.1.2.6. 高性能的序列化框架.............................................................................................................................151小包封大包，防止网络阻塞..............................................................................................................................152软中断 Hash 值和 CPU 绑定.............................................................................................................................1528.1.3. Netty RPC 实现.........................................................................................................................1528.1.3.1. 概念 ........................................................................................................................................................1528.1.3.2. 关键技术 ................................................................................................................................................1528.1.3.3. 核心流程 ................................................................................................................................................1528.1.3.1. 消息编解码.............................................................................................................................................153息数据结构（接口名称 方法名 参数类型和参数值 超时时间 requestID） ...........................................153序列化..................................................................................................................................................................1548.1.3.1. 通讯过程 ................................................................................................................................................154核心问题(线程暂停、消息乱序) .......................................................................................................................154通讯流程..............................................................................................................................................................154requestID 生成-AtomicLong .............................................................................................................................154存放回调对象 callback 到全局 ConcurrentHashMap ....................................................................................154synchronized 获取回调对象 callback 的锁并自旋 wait..................................................................................154监听消息的线程收到消息，找到 callback 上的锁并唤醒 ..............................................................................1558.1.4. RMI 实现方式............................................................................................................................1558.1.4.1. 实现步骤 ................................................................................................................................................1558.1.5. Protoclol Buffer.........................................................................................................................1568.1.5.1. 特点 ........................................................................................................................................................1578.1.6. Thrift ...........................................................................................................................................1579. 网络.....................................................................................................................................................1599.1.1. 网络 7 层架构 ............................................................................................................................1599.1.2. TCP/IP 原理...............................................................................................................................1609.1.2.1. 网络访问层(Network Access Layer)...................................................................................................1609.1.2.2. 网络层(Internet Layer) .........................................................................................................................1609.1.2.3. 传输层(Tramsport Layer-TCP/UDP) ..................................................................................................1609.1.2.4. 应用层(Application Layer)....................................................................................................................1609.1.3. TCP 三次握手/四次挥手 ..........................................................................................................1619.1.3.1. 数据包说明.............................................................................................................................................1619.1.3.2. 三次握手 ................................................................................................................................................1629.1.3.3. 四次挥手 ................................................................................................................................................1639.1.4. HTTP 原理.................................................................................................................................1649.1.4.1. 传输流程 ................................................................................................................................................1641：地址解析 .......................................................................................................................................................1642：封装 HTTP 请求数据包 ...............................................................................................................................1653：封装成 TCP 包并建立连接..........................................................................................................................1654：客户机发送请求命........................................................................................................................................1655：服务器响应....................................................................................................................................................1656：服务器关闭 TCP 连接..................................................................................................................................1659.1.4.2. HTTP 状态 .............................................................................................................................................1659.1.4.3. HTTPS ...................................................................................................................................................166建立连接获取证书..............................................................................................................................................167证书验证..............................................................................................................................................................167数据加密和传输..................................................................................................................................................1679.1.5. CDN 原理...................................................................................................................................1679.1.5.1. 分发服务系统.........................................................................................................................................1679.1.5.2. 负载均衡系统：.....................................................................................................................................1689.1.5.3. 管理系统：.............................................................................................................................................16810. 日志 .................................................................................................................................................16910.1.1. Slf4j ............................................................................................................................................16910.1.2. Log4j ..........................................................................................................................................16910.1.3. LogBack.....................................................................................................................................16910.1.3.1. Logback 优点...................................................................................................................................16910.1.4. ELK.............................................................................................................................................17013/04/2018 Page 10 of 28311. ZOOKEEPER .................................................................................................................................17111.1.1. Zookeeper 概念 ........................................................................................................................17111.1.1. Zookeeper 角色 ........................................................................................................................17111.1.1.1. Leader...............................................................................................................................................17111.1.1.2. Follower............................................................................................................................................17111.1.1.3. Observer...........................................................................................................................................17111.1.1.1. ZAB 协议 ...........................................................................................................................................172事务编号 Zxid（事务请求计数器 epoch） ...................................................................................................172epoch...................................................................................................................................................................172Zab 协议有两种模式-恢复模式（选主）、广播模式（同步）......................................................................172ZAB 协议 4 阶段.................................................................................................................................................172Leader election（选举阶段-选出准 Leader） ................................................................................................172Discovery（发现阶段-接受提议、生成 epoch、接受 epoch）....................................................................173Synchronization（同步阶段-同步 follower 副本）.........................................................................................173Broadcast（广播阶段-leader 消息广播）.......................................................................................................173ZAB 协议 JAVA 实现（FLE-发现阶段和同步合并为 Recovery Phase（恢复阶段））............................17311.1.1.2. 投票机制............................................................................................................................................17311.1.2. Zookeeper 工作原理（原子广播）.........................................................................................17411.1.3. Znode 有四种形式的目录节点 ................................................................................................17412. KAFKA............................................................................................................................................17512.1.1. Kafka 概念 .................................................................................................................................17512.1.2. Kafka 数据存储设计 .................................................................................................................17512.1.2.1. partition 的数据文件（offset，MessageSize，data）............................................................17512.1.2.2. 数据文件分段 segment（顺序读写、分段命令、二分查找）....................................................17612.1.2.3. 数据文件索引（分段索引、稀疏存储）.........................................................................................17612.1.3. 生产者设计 ................................................................................................................................17612.1.3.1. 负载均衡（partition 会均衡分布到不同 broker 上）.................................................................17612.1.3.2. 批量发送............................................................................................................................................17712.1.3.3. 压缩（GZIP 或 Snappy）...............................................................................................................17712.1.1. 消费者设计 ................................................................................................................................17712.1.1.1. Consumer Group ...........................................................................................................................17813. RABBITMQ ....................................................................................................................................17913.1.1. 概念 ............................................................................................................................................17913.1.2. RabbitMQ 架构 .........................................................................................................................17913.1.2.1. Message ...........................................................................................................................................18013.1.2.2. Publisher ..........................................................................................................................................18013.1.2.3. Exchange（将消息路由给队列 ） .................................................................................................18013.1.2.4. Binding（消息队列和交换器之间的关联）..................................................................................18013.1.2.5. Queue ...............................................................................................................................................18013.1.2.6. Connection......................................................................................................................................18013.1.2.7. Channel ............................................................................................................................................18013.1.2.8. Consumer.........................................................................................................................................18013.1.2.9. Virtual Host .....................................................................................................................................18013.1.2.10. Broker...............................................................................................................................................18113.1.3. Exchange 类型 .........................................................................................................................18113.1.3.1. Direct 键（routing key）分布： ..................................................................................................18113.1.3.2. Fanout（广播分发）.......................................................................................................................18113.1.3.3. topic 交换器（模式匹配） ...................................................................................................18213/04/2018 Page 11 of 28314. HBASE............................................................................................................................................18314.1.1. 概念 ............................................................................................................................................18314.1.2. 列式存储 ....................................................................................................................................18314.1.3. Hbase 核心概念........................................................................................................................18414.1.3.1. Column Family 列族.......................................................................................................................18414.1.3.2. Rowkey（Rowkey 查询，Rowkey 范围扫描，全表扫描）.......................................................18414.1.3.3. Region 分区......................................................................................................................................18414.1.3.4. TimeStamp 多版本..........................................................................................................................18414.1.4. Hbase 核心架构........................................................................................................................18414.1.4.1. Client：.............................................................................................................................................18514.1.4.2. Zookeeper：....................................................................................................................................18514.1.4.3. Hmaster............................................................................................................................................18514.1.4.4. HregionServer.................................................................................................................................18514.1.4.5. Region 寻址方式（通过 zookeeper .META）............................................................................18614.1.4.6. HDFS .................................................................................................................................................18614.1.5. Hbase 的写逻辑........................................................................................................................18714.1.5.1. Hbase 的写入流程 ...........................................................................................................................187获取 RegionServer ............................................................................................................................................187请求写 Hlog ........................................................................................................................................................187请求写 MemStore ..............................................................................................................................................18714.1.5.2. MemStore 刷盘...............................................................................................................................187全局内存控制......................................................................................................................................................188MemStore 达到上限...........................................................................................................................................188RegionServer 的 Hlog 数量达到上限...............................................................................................................188手工触发..............................................................................................................................................................188关闭 RegionServer 触发....................................................................................................................................188Region 使用 HLOG 恢复完数据后触发............................................................................................................18814.1.6. HBase vs Cassandra...............................................................................................................18815. MONGODB.....................................................................................................................................19015.1.1. 概念 ............................................................................................................................................19015.1.2. 特点 ............................................................................................................................................19016. CASSANDRA.................................................................................................................................19216.1.1. 概念 ............................................................................................................................................19216.1.2. 数据模型 ....................................................................................................................................192Key Space（对应 SQL 数据库中的 database）................................................................................................192Key（对应 SQL 数据库中的主键）......................................................................................................................192column（对应 SQL 数据库中的列）....................................................................................................................192super column（SQL 数据库不支持）..................................................................................................................192Standard Column Family（相对应 SQL 数据库中的 table）............................................................................192Super Column Family（SQL 数据库不支持） ...................................................................................................19216.1.3. Cassandra 一致 Hash 和虚拟节点 .........................................................................................192一致性 Hash（多米诺 down 机）.........................................................................................................................192虚拟节点（down 机多节点托管）........................................................................................................................19316.1.4. Gossip 协议...............................................................................................................................193Gossip 节点的通信方式及收敛性 .........................................................................................................................194Gossip 两个节点（A、B）之间存在三种通信方式（push、pull、push&pull）........................................194gossip 的协议和 seed list（防止集群分列） ..................................................................................................19416.1.5. 数据复制 ....................................................................................................................................194Partitioners（计算 primary key token 的 hash 函数）.......................................................................................194两种可用的复制策略： ..........................................................................................................................................194SimpleStrategy：仅用于单数据中心，...........................................................................................................194将第一个 replica 放在由 partitioner 确定的节点中，其余的 replicas 放在上述节点顺时针方向的后续节点中。..................................................................................................................................................................19413/04/2018 Page 12 of 283NetworkTopologyStrategy：可用于较复杂的多数据中心。.........................................................................194可以指定在每个数据中心分别存储多少份 replicas。...................................................................................19416.1.6. 数据写请求和协调者 ................................................................................................................195协调者(coordinator)................................................................................................................................................19516.1.7. 数据读请求和后台修复 ............................................................................................................19516.1.8. 数据存储（CommitLog、MemTable、SSTable）...........................................................196SSTable 文件构成（BloomFilter、index、data、static）................................................................................19616.1.9. 二级索引（对要索引的 value 摘要，生成 RowKey）..........................................................19616.1.10. 数据读写 ................................................................................................................................197数据写入和更新（数据追加） ..............................................................................................................................197数据的写和删除效率极高..................................................................................................................................197错误恢复简单......................................................................................................................................................197读的复杂度高......................................................................................................................................................197数据删除（column 的墓碑） ................................................................................................................................197墓碑......................................................................................................................................................................198垃圾回收 compaction ........................................................................................................................................198数据读取（memtable SStables）................................................................................................................198行缓存和键缓存请求流程图 ..................................................................................................................................199Row Cache（SSTables 中频繁被访问的数据）............................................................................................199Bloom Filter（查找数据可能对应的 SSTable）.............................................................................................200Partition Key Cache（查找数据可能对应的 Partition key） ........................................................................200Partition Summary（内存中存储一些 partition index 的样本）...................................................................200Partition Index（磁盘中） ................................................................................................................................200Compression offset map（磁盘中）...............................................................................................................20017. 设计模式..........................................................................................................................................20117.1.1. 设计原则 ....................................................................................................................................20117.1.2. 工厂方法模式 ............................................................................................................................20117.1.3. 抽象工厂模式 ............................................................................................................................20117.1.4. 单例模式 ....................................................................................................................................20117.1.5. 建造者模式 ................................................................................................................................20117.1.6. 原型模式 ....................................................................................................................................20117.1.7. 适配器模式 ................................................................................................................................20117.1.8. 装饰器模式 ................................................................................................................................20117.1.9. 代理模式 ....................................................................................................................................20117.1.10. 外观模式 ................................................................................................................................20117.1.11. 桥接模式 ................................................................................................................................20117.1.12. 组合模式 ................................................................................................................................20117.1.13. 享元模式 ................................................................................................................................20117.1.14. 策略模式 ................................................................................................................................20117.1.15. 模板方法模式 ........................................................................................................................20117.1.16. 观察者模式 ............................................................................................................................20117.1.17. 迭代子模式 ............................................................................................................................20117.1.18. 责任链模式 ............................................................................................................................20117.1.19. 命令模式 ................................................................................................................................20117.1.20. 备忘录模式 ............................................................................................................................20117.1.21. 状态模式 ................................................................................................................................20217.1.22. 访问者模式 ............................................................................................................................20217.1.23. 中介者模式 ............................................................................................................................20217.1.24. 解释器模式 ............................................................................................................................20218. 负载均衡..........................................................................................................................................20318.1.1. 四层负载均衡 vs 七层负载均衡 ..............................................................................................20318.1.1.1. 四层负载均衡（目标地址和端口交换）.........................................................................................203F5：硬件负载均衡器，功能很好，但是成本很高。......................................................................................203lvs：重量级的四层负载软件。 .........................................................................................................................203nginx：轻量级的四层负载软件，带缓存功能，正则表达式较灵活。 .........................................................20313/04/2018 Page 13 of 283haproxy：模拟四层转发，较灵活。................................................................................................................20318.1.1.2. 七层负载均衡（内容交换） ............................................................................................................203haproxy：天生负载均衡技能，全面支持七层代理，会话保持，标记，路径转移；.................................204nginx：只在 http 协议和 mail 协议上功能比较好，性能与 haproxy 差不多；............................................204apache：功能较差.............................................................................................................................................204Mysql proxy：功能尚可。.................................................................................................................................20418.1.2. 负载均衡算法/策略 ...................................................................................................................20418.1.2.1. 轮循均衡（Round Robin） ...........................................................................................................20418.1.2.2. 权重轮循均衡（Weighted Round Robin）................................................................................20418.1.2.3. 随机均衡（Random） ....................................................................................................................20418.1.2.4. 权重随机均衡（Weighted Random）.........................................................................................20418.1.2.5. 响应速度均衡（Response Time 探测时间）...............................................................................20418.1.2.6. 最少连接数均衡（Least Connection）........................................................................................20518.1.2.7. 处理能力均衡（CPU、内存） ........................................................................................................20518.1.2.8. DNS 响应均衡（Flash DNS） .......................................................................................................20518.1.2.9. 哈希算法............................................................................................................................................20518.1.2.10. IP 地址散列（保证客户端服务器对应关系稳定）........................................................................20518.1.2.11. URL 散列...........................................................................................................................................20518.1.3. LVS.............................................................................................................................................20618.1.3.1. LVS 原理............................................................................................................................................206IPVS ....................................................................................................................................................................20618.1.3.1. LVS NAT 模式 ..................................................................................................................................20718.1.3.2. LVS DR 模式（局域网改写 mac 地址）........................................................................................20818.1.3.3. LVS TUN 模式（IP 封装、跨网段）..............................................................................................20918.1.3.4. LVS FULLNAT 模式.........................................................................................................................21018.1.4. Keepalive...................................................................................................................................21118.1.5. Nginx 反向代理负载均衡 .........................................................................................................21118.1.5.1. upstream_module 和健康检测........................................................................................................21218.1.5.1. proxy_pass 请求转发.......................................................................................................................21218.1.6. HAProxy ....................................................................................................................................21319. 数据库 .............................................................................................................................................21419.1.1. 存储引擎 ....................................................................................................................................21419.1.1.1. 概念....................................................................................................................................................21419.1.1.2. InnoDB（B 树） .............................................................................................................................21419.1.1.3. TokuDB（Fractal Tree-节点带数据） ..........................................................................................21519.1.1.4. MyIASM.............................................................................................................................................21519.1.1.5. Memory..............................................................................................................................................21519.1.2. 索引 ............................................................................................................................................21519.1.2.1. 常见索引原则有 ................................................................................................................................2161.选择唯一性索引 ..............................................................................................................................................2162.为经常需要排序、分组和联合操作的字段建立索引： ...............................................................................2163．为常作为查询条件的字段建立索引。........................................................................................................2164．限制索引的数目：........................................................................................................................................216尽量使用数据量少的索引..................................................................................................................................216尽量使用前缀来索引..........................................................................................................................................2167．删除不再使用或者很少使用的索引............................................................................................................2168 . 最左前缀匹配原则，非常重要的原则。.....................................................................................................21610 . 尽量选择区分度高的列作为索引 ..............................................................................................................21611 .索引列不能参与计算，保持列“干净”：带函数的查询不参与索引。 ................................................21612 .尽量的扩展索引，不要新建索引。............................................................................................................21619.1.3. 数据库三范式 ............................................................................................................................21619.1.3.1. 第一范式(1st NF －列都是不可再分).............................................................................................21619.1.3.2. 第二范式(2nd NF－每个表只描述一件事情).................................................................................21619.1.3.3. 第三范式(3rd NF－ 不存在对非主键列的传递依赖).....................................................................21719.1.4. 数据库是事务 ............................................................................................................................21713/04/2018 Page 14 of 283原子性（Atomicity）..........................................................................................................................................217一致性（Consistency）....................................................................................................................................217隔离性（Isolation）...........................................................................................................................................218永久性（Durability） .........................................................................................................................................21819.1.5. 存储过程(特定功能的 SQL 语句集)........................................................................................218存储过程优化思路： ..............................................................................................................................................21819.1.6. 触发器(一段能自动执行的程序)..............................................................................................21819.1.7. 数据库并发策略 ........................................................................................................................21819.1.7.1. 乐观锁................................................................................................................................................21819.1.7.2. 悲观锁................................................................................................................................................21919.1.7.3. 时间戳................................................................................................................................................21919.1.8. 数据库锁 ....................................................................................................................................21919.1.8.1. 行级锁................................................................................................................................................21919.1.8.2. 表级锁................................................................................................................................................21919.1.8.1. 页级锁................................................................................................................................................21919.1.9. 基于 Redis 分布式锁 ................................................................................................................21919.1.10. 分区分表 ................................................................................................................................220垂直切分(按照功能模块) ...................................................................................................................................220水平切分(按照规则划分存储) ...........................................................................................................................22019.1.11. 两阶段提交协议 ....................................................................................................................22019.1.11.1. 准备阶段............................................................................................................................................22119.1.11.2. 提交阶段............................................................................................................................................22119.1.11.3. 缺点....................................................................................................................................................221同步阻塞问题......................................................................................................................................................221单点故障..............................................................................................................................................................221数据不一致（脑裂问题）..................................................................................................................................221二阶段无法解决的问题（数据状态不确定） ..................................................................................................22119.1.12. 三阶段提交协议 ....................................................................................................................22219.1.12.1. CanCommit 阶段 ..............................................................................................................................22219.1.12.2. PreCommit 阶段 ...............................................................................................................................22219.1.12.3. doCommit 阶段.................................................................................................................................22219.1.13. 柔性事务 ................................................................................................................................22219.1.13.1. 柔性事务............................................................................................................................................222两阶段型..............................................................................................................................................................222补偿型..................................................................................................................................................................222异步确保型..........................................................................................................................................................223最大努力通知型（多次尝试）..........................................................................................................................22319.1.14. CAP........................................................................................................................................224一致性（C）： .......................................................................................................................................................224可用性（A）： .......................................................................................................................................................224分区容忍性（P）：................................................................................................................................................22420. 一致性算法......................................................................................................................................22520.1.1. Paxos .........................................................................................................................................225Paxos 三种角色：Proposer，Acceptor，Learners ...........................................................................................225Proposer： .........................................................................................................................................................225Acceptor：..........................................................................................................................................................225Learner：............................................................................................................................................................225Paxos 算法分为两个阶段。具体如下：...............................................................................................................225阶段一（准 leader 确定 ）：............................................................................................................................225阶段二（leader 确认）：..................................................................................................................................22520.1.2. Zab .............................................................................................................................................2251.崩溃恢复：主要就是 Leader 选举过程.........................................................................................................2262.数据同步：Leader 服务器与其他服务器进行数据同步..............................................................................2263.消息广播：Leader 服务器将数据发送给其他服务器..................................................................................22620.1.3. Raft.............................................................................................................................................22620.1.3.1. 角色....................................................................................................................................................226Leader（领导者-日志管理） ............................................................................................................................226Follower（追随者-日志同步）..........................................................................................................................226Candidate（候选者-负责选票）.......................................................................................................................22613/04/2018 Page 15 of 28320.1.3.2. Term（任期）...................................................................................................................................22620.1.3.3. 选举（Election）..............................................................................................................................227选举定时器..........................................................................................................................................................22720.1.3.4. 安全性（Safety）.............................................................................................................................22720.1.3.5. raft 协议和 zab 协议区别 .................................................................................................................22720.1.4. NWR...........................................................................................................................................228N：在分布式存储系统中，有多少份备份数据................................................................................................228W：代表一次成功的更新操作要求至少有 w 份数据写入成功 ......................................................................228R： 代表一次成功的读数据操作要求至少有 R 份数据成功读取..................................................................22820.1.5. Gossip........................................................................................................................................22820.1.6. 一致性 Hash..............................................................................................................................22920.1.6.1. 一致性 Hash 特性.............................................................................................................................22920.1.6.2. 一致性 Hash 原理.............................................................................................................................2291.建构环形 hash 空间：....................................................................................................................................2292.把需要缓存的内容(对象)映射到 hash 空间..................................................................................................2293.把服务器(节点)映射到 hash 空间 .................................................................................................................2294.把对象映射到服务节点...................................................................................................................................229考察 cache 的变动.............................................................................................................................................230虚拟节点..............................................................................................................................................................23021. JAVA 算法 ......................................................................................................................................23221.1.1. 二分查找 ....................................................................................................................................23221.1.2. 冒泡排序算法 ............................................................................................................................23221.1.3. 插入排序算法 ............................................................................................................................23321.1.4. 快速排序算法 ............................................................................................................................23421.1.1. 希尔排序算法 ............................................................................................................................23621.1.2. 归并排序算法 ............................................................................................................................23721.1.3. 桶排序算法 ................................................................................................................................24021.1.4. 基数排序算法 ............................................................................................................................24121.1.5. 剪枝算法 ....................................................................................................................................24321.1.6. 回溯算法 ....................................................................................................................................24321.1.7. 最短路径算法 ............................................................................................................................24321.1.8. 最大子数组算法 ........................................................................................................................24321.1.9. 最长公共子序算法 ....................................................................................................................24321.1.10. 最小生成树算法 ....................................................................................................................24322. 数据结构..........................................................................................................................................24522.1.1. 栈（stack）...............................................................................................................................24522.1.2. 队列（queue） .........................................................................................................................24522.1.3. 链表（Link）.............................................................................................................................24522.1.4. 散列表（Hash Table）............................................................................................................24622.1.5. 排序二叉树 ................................................................................................................................24622.1.5.1. 插入操作............................................................................................................................................24622.1.5.2. 删除操作............................................................................................................................................24722.1.5.3. 查询操作............................................................................................................................................24822.1.6. 红黑树 ........................................................................................................................................24822.1.6.1. 红黑树的特性....................................................................................................................................24822.1.6.1. 左旋....................................................................................................................................................24822.1.6.1. 右旋....................................................................................................................................................24922.1.6.1. 添加....................................................................................................................................................25022.1.6.2. 删除....................................................................................................................................................25122.1.7. B-TREE......................................................................................................................................25222.1.8. 位图 ............................................................................................................................................25423. 加密算法..........................................................................................................................................25523.1.1. AES ............................................................................................................................................25523.1.2. RSA............................................................................................................................................25523.1.3. CRC............................................................................................................................................25623.1.4. MD5............................................................................................................................................25613/04/2018 Page 16 of 28324. 分布式缓存......................................................................................................................................25724.1.1. 缓存雪崩 ....................................................................................................................................25724.1.2. 缓存穿透 ....................................................................................................................................25724.1.3. 缓存预热 ....................................................................................................................................25724.1.4. 缓存更新 ....................................................................................................................................25724.1.5. 缓存降级 ....................................................................................................................................25725. HADOOP ........................................................................................................................................25925.1.1. 概念 ............................................................................................................................................25925.1.2. HDFS .........................................................................................................................................25925.1.2.1. Client..................................................................................................................................................25925.1.2.2. NameNode........................................................................................................................................25925.1.2.3. Secondary NameNode .................................................................................................................25925.1.2.4. DataNode.........................................................................................................................................25925.1.3. MapReduce...............................................................................................................................26025.1.3.1. Client.................................................................................................................................................26025.1.3.2. JobTracker .......................................................................................................................................26025.1.3.3. TaskTracker......................................................................................................................................26125.1.3.4. Task ...................................................................................................................................................26125.1.3.5. Reduce Task 执行过程 ...................................................................................................................26125.1.4. Hadoop MapReduce 作业的生命周期...................................................................................2621.作业提交与初始化...........................................................................................................................................2622.任务调度与监控。...........................................................................................................................................2623.任务运行环境准备...........................................................................................................................................2624.任务执行 ..........................................................................................................................................................2625.作业完成。 ......................................................................................................................................................26226. SPARK............................................................................................................................................26326.1.1. 概念 ............................................................................................................................................26326.1.2. 核心架构 ....................................................................................................................................263Spark Core .........................................................................................................................................................263Spark SQL ..........................................................................................................................................................263Spark Streaming................................................................................................................................................263Mllib .....................................................................................................................................................................263GraphX................................................................................................................................................................26326.1.3. 核心组件 ....................................................................................................................................264Cluster Manager-制整个集群，监控 worker .................................................................................................264Worker 节点-负责控制计算节点.......................................................................................................................264Driver： 运行 Application 的 main()函数.........................................................................................................264Executor：执行器，是为某个 Application 运行在 worker node 上的一个进程..........................................26426.1.4. SPARK 编程模型......................................................................................................................26426.1.5. SPARK 计算模型......................................................................................................................26526.1.6. SPARK 运行流程......................................................................................................................2661. 构建 Spark Application 的运行环境，启动 SparkContext....................................................................2672. SparkContext 向资源管理器（可以是 Standalone，Mesos，Yarn）申请运行 Executor 资源，并启动 StandaloneExecutorbackend，..................................................................................................................2673. Executor 向 SparkContext 申请 Task .....................................................................................................2674. SparkContext 将应用程序分发给 Executor............................................................................................2675. SparkContext 构建成 DAG 图，将 DAG 图分解成 Stage、将 Taskset 发送给 Task Scheduler，最后由 Task Scheduler 将 Task 发送给 Executor 运行.....................................................................................2676. Task 在 Executor 上运行，运行完释放所有资源...................................................................................26726.1.7. SPARK RDD 流程 ....................................................................................................................26726.1.8. SPARK RDD.............................................................................................................................267（1）RDD 的创建方式...........................................................................................................................................267（2）RDD 的两种操作算子（转换（Transformation）与行动（Action））..............................................26827. STORM ...........................................................................................................................................26913/04/2018 Page 17 of 28327.1.1. 概念 ............................................................................................................................................26927.1.1. 集群架构 ....................................................................................................................................26927.1.1.1. Nimbus（master-代码分发给 Supervisor） ................................................................................26927.1.1.2. Supervisor（slave-管理 Worker 进程的启动和终止）...............................................................26927.1.1.3. Worker（具体处理组件逻辑的进程）............................................................................................26927.1.1.4. Task ...................................................................................................................................................27027.1.1.5. ZooKeeper ........................................................................................................................................27027.1.2. 编程模型（spout->tuple->bolt）.......................................................................................27027.1.2.1. Topology............................................................................................................................................27027.1.2.2. Spout..................................................................................................................................................27027.1.2.3. Bolt.....................................................................................................................................................27027.1.2.4. Tuple..................................................................................................................................................27027.1.2.5. Stream ...............................................................................................................................................27127.1.3. Topology 运行...........................................................................................................................271(1). Worker（进程） (2). Executor（线程） (3). Task..................................................................................27127.1.3.1. Worker(1 个 worker 进程执行的是 1 个 topology 的子集) .......................................................27127.1.3.2. Executor(executor 是 1 个被 worker 进程启动的单独线程)......................................................27127.1.3.3. Task(最终运行 spout 或 bolt 中代码的单元)...............................................................................27227.1.4. Storm Streaming Grouping.....................................................................................................27227.1.4.1. huffle Grouping.................................................................................................................................27327.1.4.2. Fields Grouping................................................................................................................................27327.1.4.3. All grouping ：广播..........................................................................................................................27327.1.4.4. Global grouping................................................................................................................................27427.1.4.5. None grouping ：不分组.................................................................................................................27427.1.4.6. Direct grouping ：直接分组 指定分组 ...........................................................................................27428. YARN ..............................................................................................................................................27528.1.1. 概念 ............................................................................................................................................27528.1.2. ResourceManager ...................................................................................................................27528.1.3. NodeManager...........................................................................................................................27528.1.4. ApplicationMaster .................................................................................................................27628.1.5. YARN 运行流程 ....................................................................................................................27729. 机器学习..........................................................................................................................................27829.1.1. 决策树 ........................................................................................................................................27829.1.2. 随机森林算法 ............................................................................................................................27829.1.3. 逻辑回归 ....................................................................................................................................27829.1.4. SVM............................................................................................................................................27829.1.5. 朴素贝叶斯 ................................................................................................................................27829.1.6. K 最近邻算法.............................................................................................................................27829.1.7. K 均值算法.................................................................................................................................27829.1.8. Adaboost 算法 ..........................................................................................................................27829.1.9. 神经网络 ....................................................................................................................................27829.1.10. 马尔可夫 ................................................................................................................................27830. 云计算 .............................................................................................................................................27930.1.1. SaaS ..........................................................................................................................................27930.1.2. PaaS ..........................................................................................................................................27930.1.3. IaaS............................................................................................................................................27930.1.4. Docker........................................................................................................................................27930.1.4.1. 概念....................................................................................................................................................27930.1.4.2. Namespaces.....................................................................................................................................28030.1.4.3. 进程(CLONE_NEWPID 实现的进程隔离)......................................................................................28130.1.4.4. Libnetwork 与网络隔离....................................................................................................................28130.1.4.5. 资源隔离与 CGroups .......................................................................................................................28230.1.4.6. 镜像与 UnionFS................................................................................................................................28230.1.4.7. 存储驱动............................................................................................................................................28213/04/2018 Page 18 of 28330.1.5. Openstack .................................................................................................................................283