在采用分库分表的数据库结构设计时,往数据库中新增数据(insert)不能在通过自增id来保证id唯一了,因为分表两个同样的表在不同服务器上自增id会重复,所有必须通过手动添加id来保证id的唯一性,snowflake (雪花)算法(twitter出品)就是用来生成唯一主键值很好的选择
一、分库分表设计 1.分库
就是为每个模块单独建立一个数据库,在数据不多的情况下,数据库可以集中在同一个服务器上,可以节约成本,当某个数据库数据比较多时,则可以把该数据库单独迁移到一台服务器上提高性能
2.分表
当某个数据库数据非常多单台服务器也不能承载这么多数据时,则可以在新的服务器创建一样数据库和表,进行数据存储,但是分表之后要解决数据存储和查询等问题,比如新存入一个数据要存到哪个服务器上
mysql支持的数据是百万级别,而orcal支持的是千万级别,比如当mysql数据的索引超过百万时,性能方面会下降严重
3.分库分表插件
二、雪花算法 1.概述
41bit 的时间戳可以支持该算法使用到2082年
10bit 的工作机器id可以支持1024台机器
序列号支持 1毫秒产生4096个自增序列id
整体上按照时间自增排序
整个分布式系统内不会产生 ID碰撞
每秒能够产生 26万ID左右
2.IdWorker工具类(分布式id生成器) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 import java.lang.management.ManagementFactory;import java.net.InetAddress;import java.net.NetworkInterface;public class IdWorker { private final static long twepoch = 1288834974657L ; private final static long workerIdBits = 5L ; private final static long datacenterIdBits = 5L ; private final static long maxWorkerId = -1L ^ (-1L << workerIdBits); private final static long maxDatacenterId = -1L ^ (-1L << datacenterIdBits); private final static long sequenceBits = 12L ; private final static long workerIdShift = sequenceBits; private final static long datacenterIdShift = sequenceBits + workerIdBits; private final static long timestampLeftShift = sequenceBits + workerIdBits + datacenterIdBits; private final static long sequenceMask = -1L ^ (-1L << sequenceBits); private static long lastTimestamp = -1L ; private long sequence = 0L ; private final long workerId; private final long datacenterId; public IdWorker () { this .datacenterId = getDatacenterId(maxDatacenterId); this .workerId = getMaxWorkerId(datacenterId, maxWorkerId); } public IdWorker (long workerId, long datacenterId) { if (workerId > maxWorkerId || workerId < 0 ) { throw new IllegalArgumentException(String.format("worker Id can't be greater than %d or less than 0" , maxWorkerId)); } if (datacenterId > maxDatacenterId || datacenterId < 0 ) { throw new IllegalArgumentException(String.format("datacenter Id can't be greater than %d or less than 0" , maxDatacenterId)); } this .workerId = workerId; this .datacenterId = datacenterId; } public synchronized long nextId () { long timestamp = timeGen(); if (timestamp < lastTimestamp) { throw new RuntimeException(String.format("Clock moved backwards. Refusing to generate id for %d milliseconds" , lastTimestamp - timestamp)); } if (lastTimestamp == timestamp) { sequence = (sequence + 1 ) & sequenceMask; if (sequence == 0 ) { timestamp = tilNextMillis(lastTimestamp); } } else { sequence = 0L ; } lastTimestamp = timestamp; long nextId = ((timestamp - twepoch) << timestampLeftShift) | (datacenterId << datacenterIdShift) | (workerId << workerIdShift) | sequence; return nextId; } private long tilNextMillis (final long lastTimestamp) { long timestamp = this .timeGen(); while (timestamp <= lastTimestamp) { timestamp = this .timeGen(); } return timestamp; } private long timeGen () { return System.currentTimeMillis(); } protected static long getMaxWorkerId (long datacenterId, long maxWorkerId) { StringBuffer mpid = new StringBuffer(); mpid.append(datacenterId); String name = ManagementFactory.getRuntimeMXBean().getName(); if (!name.isEmpty()) { mpid.append(name.split("@" )[0 ]); } return (mpid.toString().hashCode() & 0xffff ) % (maxWorkerId + 1 ); } protected static long getDatacenterId (long maxDatacenterId) { long id = 0L ; try { InetAddress ip = InetAddress.getLocalHost(); NetworkInterface network = NetworkInterface.getByInetAddress(ip); if (network == null ) { id = 1L ; } else { byte [] mac = network.getHardwareAddress(); id = ((0x000000FF & (long ) mac[mac.length - 1 ]) | (0x0000FF00 & (((long ) mac[mac.length - 2 ]) << 8 ))) >> 6 ; id = id % (maxDatacenterId + 1 ); } } catch (Exception e) { System.out.println(" getDatacenterId: " + e.getMessage()); } return id; } public static void main (String[] args) { IdWorker idWorker = new IdWorker(0 ,0 ); for (int i = 0 ; i <10000 ; i++) { long nextId=idWorker.nextId(); System.out.println(nextId); } } }
3.分布式id生成器的使用
1 2 3 4 5 6 7 8 9 @Bean public IdWorker createIdWorker () { return new IdWorker (1 ,1 ); }
在service层生成分布式id,并且在调用dao插入数据之前替换原来的id
1 2 3 4 5 6 public void save (Article article) { String id = idWorker.nextId ( ) + "" ; article.setId ( id ); articleDao.insert ( article ); }