2013-04-04 5 views
2

J'expérimente avec Kyoto Cabinet HashDB. Ma base de données atteindra la taille de 110 To. Les clés sont 20 octets et les valeurs sont 128 Ko. J'essaie d'insérer environ 500 millions de dossiers. Voici la meilleure performance que j'ai eue jusqu'à présent:Kyoto Cabinet HashDB inserts ne sont pas O (1)

Le temps est en secondes. 'n' est le nombre de 10000 entrées entrées. Par exemple 1 10000 enregistrements ont pris 1 seconde. 5e 10000 enregistrements ont pris 5 secondes et ainsi de suite. Comme vous pouvez le voir, la performance commence à se dégrader après environ 2 millions d'entrées. J'ai créé la base de données avec le nombre de seau fixé à 7 milliards, et tout le reste est par défaut. J'utilise la bibliothèque C/C++, et la définition de la capacité de la carte mémoire à 30 Go (j'ai 32 Go de RAM) en utilisant db.tune_map(30LL << 30);.

Inserted 10000 records. n: 1. Time: 1 
Inserted 10000 records. n: 2. Time: 1 
Inserted 10000 records. n: 3. Time: 1 
Inserted 10000 records. n: 4. Time: 1 
Inserted 10000 records. n: 5. Time: 4 
Inserted 10000 records. n: 6. Time: 4 
Inserted 10000 records. n: 7. Time: 4 
Inserted 10000 records. n: 8. Time: 4 
Inserted 10000 records. n: 9. Time: 4 
Inserted 10000 records. n: 10. Time: 4 
Inserted 10000 records. n: 11. Time: 4 
Inserted 10000 records. n: 12. Time: 4 
Inserted 10000 records. n: 13. Time: 4 
Inserted 10000 records. n: 14. Time: 4 
Inserted 10000 records. n: 15. Time: 4 
Inserted 10000 records. n: 16. Time: 4 
Inserted 10000 records. n: 17. Time: 4 
Inserted 10000 records. n: 18. Time: 4 
Inserted 10000 records. n: 19. Time: 7 
Inserted 10000 records. n: 20. Time: 4 
Inserted 10000 records. n: 21. Time: 4 
Inserted 10000 records. n: 22. Time: 4 
Inserted 10000 records. n: 23. Time: 5 
Inserted 10000 records. n: 24. Time: 4 
Inserted 10000 records. n: 25. Time: 4 
Inserted 10000 records. n: 26. Time: 4 
Inserted 10000 records. n: 27. Time: 4 
Inserted 10000 records. n: 28. Time: 4 
Inserted 10000 records. n: 29. Time: 5 
Inserted 10000 records. n: 30. Time: 4 
Inserted 10000 records. n: 31. Time: 4 
Inserted 10000 records. n: 32. Time: 4 
Inserted 10000 records. n: 33. Time: 4 
Inserted 10000 records. n: 34. Time: 4 
Inserted 10000 records. n: 35. Time: 4 
Inserted 10000 records. n: 36. Time: 4 
Inserted 10000 records. n: 37. Time: 5 
Inserted 10000 records. n: 38. Time: 4 
Inserted 10000 records. n: 39. Time: 4 
Inserted 10000 records. n: 40. Time: 4 
Inserted 10000 records. n: 41. Time: 4 
Inserted 10000 records. n: 42. Time: 4 
Inserted 10000 records. n: 43. Time: 4 
Inserted 10000 records. n: 44. Time: 4 
Inserted 10000 records. n: 45. Time: 4 
Inserted 10000 records. n: 46. Time: 4 
Inserted 10000 records. n: 47. Time: 4 
Inserted 10000 records. n: 48. Time: 5 
Inserted 10000 records. n: 49. Time: 4 
Inserted 10000 records. n: 50. Time: 4 
Inserted 10000 records. n: 51. Time: 5 
Inserted 10000 records. n: 52. Time: 4 
Inserted 10000 records. n: 53. Time: 5 
Inserted 10000 records. n: 54. Time: 5 
Inserted 10000 records. n: 55. Time: 4 
Inserted 10000 records. n: 56. Time: 5 
Inserted 10000 records. n: 57. Time: 5 
Inserted 10000 records. n: 58. Time: 5 
Inserted 10000 records. n: 59. Time: 5 
Inserted 10000 records. n: 60. Time: 5 
Inserted 10000 records. n: 61. Time: 4 
Inserted 10000 records. n: 62. Time: 5 
Inserted 10000 records. n: 63. Time: 5 
Inserted 10000 records. n: 64. Time: 4 
Inserted 10000 records. n: 65. Time: 6 
Inserted 10000 records. n: 66. Time: 4 
Inserted 10000 records. n: 67. Time: 4 
Inserted 10000 records. n: 68. Time: 7 
Inserted 10000 records. n: 69. Time: 6 
Inserted 10000 records. n: 70. Time: 7 
Inserted 10000 records. n: 71. Time: 7 
Inserted 10000 records. n: 72. Time: 6 
Inserted 10000 records. n: 73. Time: 10 
Inserted 10000 records. n: 74. Time: 7 
Inserted 10000 records. n: 75. Time: 10 
Inserted 10000 records. n: 76. Time: 9 
Inserted 10000 records. n: 77. Time: 8 
Inserted 10000 records. n: 78. Time: 10 
Inserted 10000 records. n: 79. Time: 8 
Inserted 10000 records. n: 80. Time: 6 
Inserted 10000 records. n: 81. Time: 8 
Inserted 10000 records. n: 82. Time: 8 
Inserted 10000 records. n: 83. Time: 9 
Inserted 10000 records. n: 84. Time: 9 
Inserted 10000 records. n: 85. Time: 9 
Inserted 10000 records. n: 86. Time: 9 
Inserted 10000 records. n: 87. Time: 7 
Inserted 10000 records. n: 88. Time: 10 
Inserted 10000 records. n: 89. Time: 8 
Inserted 10000 records. n: 90. Time: 10 
Inserted 10000 records. n: 91. Time: 10 
Inserted 10000 records. n: 92. Time: 10 
Inserted 10000 records. n: 93. Time: 10 
Inserted 10000 records. n: 94. Time: 10 
Inserted 10000 records. n: 95. Time: 10 
Inserted 10000 records. n: 96. Time: 11 
Inserted 10000 records. n: 97. Time: 9 
Inserted 10000 records. n: 98. Time: 10 
Inserted 10000 records. n: 99. Time: 10 
Inserted 10000 records. n: 100. Time: 11 
Inserted 10000 records. n: 101. Time: 10 
Inserted 10000 records. n: 102. Time: 11 
Inserted 10000 records. n: 103. Time: 11 
Inserted 10000 records. n: 104. Time: 11 
Inserted 10000 records. n: 105. Time: 12 
Inserted 10000 records. n: 106. Time: 11 
Inserted 10000 records. n: 107. Time: 12 
Inserted 10000 records. n: 108. Time: 12 
Inserted 10000 records. n: 109. Time: 10 
Inserted 10000 records. n: 110. Time: 11 
Inserted 10000 records. n: 111. Time: 10 
Inserted 10000 records. n: 112. Time: 10 
Inserted 10000 records. n: 113. Time: 11 
Inserted 10000 records. n: 114. Time: 10 
Inserted 10000 records. n: 115. Time: 11 
Inserted 10000 records. n: 116. Time: 10 
Inserted 10000 records. n: 117. Time: 10 
Inserted 10000 records. n: 118. Time: 10 
Inserted 10000 records. n: 119. Time: 10 
Inserted 10000 records. n: 120. Time: 11 
Inserted 10000 records. n: 121. Time: 10 
Inserted 10000 records. n: 122. Time: 11 
Inserted 10000 records. n: 123. Time: 9 
Inserted 10000 records. n: 124. Time: 11 
Inserted 10000 records. n: 125. Time: 9 
Inserted 10000 records. n: 126. Time: 11 
Inserted 10000 records. n: 127. Time: 10 
Inserted 10000 records. n: 128. Time: 10 
Inserted 10000 records. n: 129. Time: 12 
Inserted 10000 records. n: 130. Time: 10 
Inserted 10000 records. n: 131. Time: 10 
Inserted 10000 records. n: 132. Time: 10 
Inserted 10000 records. n: 133. Time: 10 
Inserted 10000 records. n: 134. Time: 16 
Inserted 10000 records. n: 135. Time: 10 
Inserted 10000 records. n: 136. Time: 10 
Inserted 10000 records. n: 137. Time: 10 
Inserted 10000 records. n: 138. Time: 10 
Inserted 10000 records. n: 139. Time: 9 
Inserted 10000 records. n: 140. Time: 10 
Inserted 10000 records. n: 141. Time: 10 
Inserted 10000 records. n: 142. Time: 10 
Inserted 10000 records. n: 143. Time: 10 
Inserted 10000 records. n: 144. Time: 11 
Inserted 10000 records. n: 145. Time: 10 
Inserted 10000 records. n: 146. Time: 11 
Inserted 10000 records. n: 147. Time: 9 
Inserted 10000 records. n: 148. Time: 10 
Inserted 10000 records. n: 149. Time: 9 
Inserted 10000 records. n: 150. Time: 10 
Inserted 10000 records. n: 151. Time: 20 
Inserted 10000 records. n: 152. Time: 18 
Inserted 10000 records. n: 153. Time: 33 
Inserted 10000 records. n: 154. Time: 25 
Inserted 10000 records. n: 155. Time: 30 
Inserted 10000 records. n: 156. Time: 18 
Inserted 10000 records. n: 157. Time: 12 
Inserted 10000 records. n: 158. Time: 13 
Inserted 10000 records. n: 159. Time: 14 
Inserted 10000 records. n: 160. Time: 10 
Inserted 10000 records. n: 161. Time: 9 
Inserted 10000 records. n: 162. Time: 12 
Inserted 10000 records. n: 163. Time: 11 
Inserted 10000 records. n: 164. Time: 11 
Inserted 10000 records. n: 165. Time: 11 
Inserted 10000 records. n: 166. Time: 11 
Inserted 10000 records. n: 167. Time: 12 
Inserted 10000 records. n: 168. Time: 10 
Inserted 10000 records. n: 169. Time: 11 
Inserted 10000 records. n: 170. Time: 12 
Inserted 10000 records. n: 171. Time: 11 
Inserted 10000 records. n: 172. Time: 10 
Inserted 10000 records. n: 173. Time: 11 
Inserted 10000 records. n: 174. Time: 11 
Inserted 10000 records. n: 175. Time: 10 
Inserted 10000 records. n: 176. Time: 11 
Inserted 10000 records. n: 177. Time: 11 
Inserted 10000 records. n: 178. Time: 10 
Inserted 10000 records. n: 179. Time: 11 
Inserted 10000 records. n: 180. Time: 11 
Inserted 10000 records. n: 181. Time: 11 
Inserted 10000 records. n: 182. Time: 10 
Inserted 10000 records. n: 183. Time: 11 
Inserted 10000 records. n: 184. Time: 11 
Inserted 10000 records. n: 185. Time: 11 
Inserted 10000 records. n: 186. Time: 11 
Inserted 10000 records. n: 187. Time: 11 
Inserted 10000 records. n: 188. Time: 11 
Inserted 10000 records. n: 189. Time: 11 
Inserted 10000 records. n: 190. Time: 11 
Inserted 10000 records. n: 191. Time: 11 
Inserted 10000 records. n: 192. Time: 13 
Inserted 10000 records. n: 193. Time: 13 
Inserted 10000 records. n: 194. Time: 13 
Inserted 10000 records. n: 195. Time: 12 
Inserted 10000 records. n: 196. Time: 13 
Inserted 10000 records. n: 197. Time: 11 
Inserted 10000 records. n: 198. Time: 11 
Inserted 10000 records. n: 199. Time: 12 
Inserted 10000 records. n: 200. Time: 11 
Inserted 10000 records. n: 201. Time: 11 
Inserted 10000 records. n: 202. Time: 11 
Inserted 10000 records. n: 203. Time: 11 
Inserted 10000 records. n: 204. Time: 11 
Inserted 10000 records. n: 205. Time: 12 
Inserted 10000 records. n: 206. Time: 11 
Inserted 10000 records. n: 207. Time: 10 
Inserted 10000 records. n: 208. Time: 10 
Inserted 10000 records. n: 209. Time: 11 
Inserted 10000 records. n: 210. Time: 10 
Inserted 10000 records. n: 211. Time: 10 
Inserted 10000 records. n: 212. Time: 11 
Inserted 10000 records. n: 213. Time: 11 
Inserted 10000 records. n: 214. Time: 10 
Inserted 10000 records. n: 215. Time: 11 
Inserted 10000 records. n: 216. Time: 11 
Inserted 10000 records. n: 217. Time: 12 
Inserted 10000 records. n: 218. Time: 12 
Inserted 10000 records. n: 219. Time: 12 
Inserted 10000 records. n: 220. Time: 11 
Inserted 10000 records. n: 221. Time: 12 
Inserted 10000 records. n: 222. Time: 10 
Inserted 10000 records. n: 223. Time: 11 
Inserted 10000 records. n: 224. Time: 11 
Inserted 10000 records. n: 225. Time: 11 
Inserted 10000 records. n: 226. Time: 11 
Inserted 10000 records. n: 227. Time: 11 
Inserted 10000 records. n: 228. Time: 11 
Inserted 10000 records. n: 229. Time: 11 
Inserted 10000 records. n: 230. Time: 11 
Inserted 10000 records. n: 231. Time: 10 
Inserted 10000 records. n: 232. Time: 11 
Inserted 10000 records. n: 233. Time: 10 
Inserted 10000 records. n: 234. Time: 11 
Inserted 10000 records. n: 235. Time: 11 
Inserted 10000 records. n: 236. Time: 11 
Inserted 10000 records. n: 237. Time: 11 
Inserted 10000 records. n: 238. Time: 10 
Inserted 10000 records. n: 239. Time: 11 
Inserted 10000 records. n: 240. Time: 11 
Inserted 10000 records. n: 241. Time: 11 
Inserted 10000 records. n: 242. Time: 11 
Inserted 10000 records. n: 243. Time: 11 
Inserted 10000 records. n: 244. Time: 11 
Inserted 10000 records. n: 245. Time: 11 
Inserted 10000 records. n: 246. Time: 12 
Inserted 10000 records. n: 247. Time: 11 
Inserted 10000 records. n: 248. Time: 12 
Inserted 10000 records. n: 249. Time: 12 
Inserted 10000 records. n: 250. Time: 11 
Inserted 10000 records. n: 251. Time: 12 
Inserted 10000 records. n: 252. Time: 12 
Inserted 10000 records. n: 253. Time: 13 
Inserted 10000 records. n: 254. Time: 10 
Inserted 10000 records. n: 255. Time: 12 
Inserted 10000 records. n: 256. Time: 12 
Inserted 10000 records. n: 257. Time: 13 
Inserted 10000 records. n: 258. Time: 11 
Inserted 10000 records. n: 259. Time: 10 
Inserted 10000 records. n: 260. Time: 12 
Inserted 10000 records. n: 261. Time: 11 
Inserted 10000 records. n: 262. Time: 11 
Inserted 10000 records. n: 263. Time: 12 
Inserted 10000 records. n: 264. Time: 10 
Inserted 10000 records. n: 265. Time: 10 
Inserted 10000 records. n: 266. Time: 11 
Inserted 10000 records. n: 267. Time: 10 
Inserted 10000 records. n: 268. Time: 11 
Inserted 10000 records. n: 269. Time: 12 
Inserted 10000 records. n: 270. Time: 12 
Inserted 10000 records. n: 271. Time: 11 
Inserted 10000 records. n: 272. Time: 11 
Inserted 10000 records. n: 273. Time: 10 
Inserted 10000 records. n: 274. Time: 12 
Inserted 10000 records. n: 275. Time: 9 
Inserted 10000 records. n: 276. Time: 12 
Inserted 10000 records. n: 277. Time: 11 
Inserted 10000 records. n: 278. Time: 11 
Inserted 10000 records. n: 279. Time: 11 
Inserted 10000 records. n: 280. Time: 10 
Inserted 10000 records. n: 281. Time: 11 
Inserted 10000 records. n: 282. Time: 11 
Inserted 10000 records. n: 283. Time: 10 
Inserted 10000 records. n: 284. Time: 10 
Inserted 10000 records. n: 285. Time: 11 
Inserted 10000 records. n: 286. Time: 10 
Inserted 10000 records. n: 287. Time: 10 
Inserted 10000 records. n: 288. Time: 11 
Inserted 10000 records. n: 289. Time: 10 
Inserted 10000 records. n: 290. Time: 9 
Inserted 10000 records. n: 291. Time: 16 
Inserted 10000 records. n: 292. Time: 10 
Inserted 10000 records. n: 293. Time: 9 
Inserted 10000 records. n: 294. Time: 11 
Inserted 10000 records. n: 295. Time: 9 
Inserted 10000 records. n: 296. Time: 9 
Inserted 10000 records. n: 297. Time: 11 
Inserted 10000 records. n: 298. Time: 39 
Inserted 10000 records. n: 299. Time: 52 
Inserted 10000 records. n: 300. Time: 54 
Inserted 10000 records. n: 301. Time: 42 
Inserted 10000 records. n: 302. Time: 42 
Inserted 10000 records. n: 303. Time: 11 
Inserted 10000 records. n: 304. Time: 9 
Inserted 10000 records. n: 305. Time: 7 
Inserted 10000 records. n: 306. Time: 8 
Inserted 10000 records. n: 307. Time: 10 
Inserted 10000 records. n: 308. Time: 10 
Inserted 10000 records. n: 309. Time: 10 
Inserted 10000 records. n: 310. Time: 10 
Inserted 10000 records. n: 311. Time: 9 
Inserted 10000 records. n: 312. Time: 11 
Inserted 10000 records. n: 313. Time: 11 
Inserted 10000 records. n: 314. Time: 13 
Inserted 10000 records. n: 315. Time: 13 
Inserted 10000 records. n: 316. Time: 12 
Inserted 10000 records. n: 317. Time: 11 
Inserted 10000 records. n: 318. Time: 10 
Inserted 10000 records. n: 319. Time: 9 
Inserted 10000 records. n: 320. Time: 12 
Inserted 10000 records. n: 321. Time: 11 
Inserted 10000 records. n: 322. Time: 12 
Inserted 10000 records. n: 323. Time: 10 
Inserted 10000 records. n: 324. Time: 11 
Inserted 10000 records. n: 325. Time: 11 
Inserted 10000 records. n: 326. Time: 13 
Inserted 10000 records. n: 327. Time: 13 
Inserted 10000 records. n: 328. Time: 12 
Inserted 10000 records. n: 329. Time: 12 
Inserted 10000 records. n: 330. Time: 10 
Inserted 10000 records. n: 331. Time: 11 
Inserted 10000 records. n: 332. Time: 10 
Inserted 10000 records. n: 333. Time: 11 
Inserted 10000 records. n: 334. Time: 11 
Inserted 10000 records. n: 335. Time: 11 
Inserted 10000 records. n: 336. Time: 10 
Inserted 10000 records. n: 337. Time: 11 
Inserted 10000 records. n: 338. Time: 12 
Inserted 10000 records. n: 339. Time: 13 
Inserted 10000 records. n: 340. Time: 14 
Inserted 10000 records. n: 341. Time: 13 
Inserted 10000 records. n: 342. Time: 10 
Inserted 10000 records. n: 343. Time: 10 
Inserted 10000 records. n: 344. Time: 9 
Inserted 10000 records. n: 345. Time: 11 
Inserted 10000 records. n: 346. Time: 12 
Inserted 10000 records. n: 347. Time: 12 
Inserted 10000 records. n: 348. Time: 10 
Inserted 10000 records. n: 349. Time: 11 
Inserted 10000 records. n: 350. Time: 13 
Inserted 10000 records. n: 351. Time: 13 
Inserted 10000 records. n: 352. Time: 11 
Inserted 10000 records. n: 353. Time: 12 
Inserted 10000 records. n: 354. Time: 10 
Inserted 10000 records. n: 355. Time: 10 
Inserted 10000 records. n: 356. Time: 12 
Inserted 10000 records. n: 357. Time: 11 
Inserted 10000 records. n: 358. Time: 11 
Inserted 10000 records. n: 359. Time: 12 
Inserted 10000 records. n: 360. Time: 11 
Inserted 10000 records. n: 361. Time: 12 
Inserted 10000 records. n: 362. Time: 13 
Inserted 10000 records. n: 363. Time: 12 
Inserted 10000 records. n: 364. Time: 11 
Inserted 10000 records. n: 365. Time: 11 
Inserted 10000 records. n: 366. Time: 11 
Inserted 10000 records. n: 367. Time: 11 
Inserted 10000 records. n: 368. Time: 12 
Inserted 10000 records. n: 369. Time: 11 
Inserted 10000 records. n: 370. Time: 14 
Inserted 10000 records. n: 371. Time: 14 
Inserted 10000 records. n: 372. Time: 15 
Inserted 10000 records. n: 373. Time: 16 
Inserted 10000 records. n: 374. Time: 18 
Inserted 10000 records. n: 375. Time: 15 
Inserted 10000 records. n: 376. Time: 11 
Inserted 10000 records. n: 377. Time: 11 
Inserted 10000 records. n: 378. Time: 11 
Inserted 10000 records. n: 379. Time: 12 
Inserted 10000 records. n: 380. Time: 11 
Inserted 10000 records. n: 381. Time: 12 
Inserted 10000 records. n: 382. Time: 12 
Inserted 10000 records. n: 383. Time: 14 
Inserted 10000 records. n: 384. Time: 12 
Inserted 10000 records. n: 385. Time: 16 
Inserted 10000 records. n: 386. Time: 14 
Inserted 10000 records. n: 387. Time: 10 
Inserted 10000 records. n: 388. Time: 12 
Inserted 10000 records. n: 389. Time: 11 
Inserted 10000 records. n: 390. Time: 11 
Inserted 10000 records. n: 391. Time: 11 
Inserted 10000 records. n: 392. Time: 12 
Inserted 10000 records. n: 393. Time: 15 
Inserted 10000 records. n: 394. Time: 16 
Inserted 10000 records. n: 395. Time: 17 
Inserted 10000 records. n: 396. Time: 13 
Inserted 10000 records. n: 397. Time: 13 
Inserted 10000 records. n: 398. Time: 17 
Inserted 10000 records. n: 399. Time: 11 
Inserted 10000 records. n: 400. Time: 16 
Inserted 10000 records. n: 401. Time: 12 
Inserted 10000 records. n: 402. Time: 13 
Inserted 10000 records. n: 403. Time: 13 
Inserted 10000 records. n: 404. Time: 13 
Inserted 10000 records. n: 405. Time: 16 
Inserted 10000 records. n: 406. Time: 10 
Inserted 10000 records. n: 407. Time: 12 
Inserted 10000 records. n: 408. Time: 14 
Inserted 10000 records. n: 409. Time: 12 
Inserted 10000 records. n: 410. Time: 18 
Inserted 10000 records. n: 411. Time: 13 
Inserted 10000 records. n: 412. Time: 20 
Inserted 10000 records. n: 413. Time: 14 
Inserted 10000 records. n: 414. Time: 15 
Inserted 10000 records. n: 415. Time: 52 
Inserted 10000 records. n: 416. Time: 22 
Inserted 10000 records. n: 417. Time: 111 
Inserted 10000 records. n: 418. Time: 81 
Inserted 10000 records. n: 419. Time: 80 
Inserted 10000 records. n: 420. Time: 78 
Inserted 10000 records. n: 421. Time: 116 
Inserted 10000 records. n: 422. Time: 123 
Inserted 10000 records. n: 423. Time: 86 
Inserted 10000 records. n: 424. Time: 24 
Inserted 10000 records. n: 425. Time: 36 
Inserted 10000 records. n: 426. Time: 21 
Inserted 10000 records. n: 427. Time: 15 
Inserted 10000 records. n: 428. Time: 17 
Inserted 10000 records. n: 429. Time: 15 
Inserted 10000 records. n: 430. Time: 15 
Inserted 10000 records. n: 431. Time: 16 
Inserted 10000 records. n: 432. Time: 15 
Inserted 10000 records. n: 433. Time: 18 
Inserted 10000 records. n: 434. Time: 20 
Inserted 10000 records. n: 435. Time: 18 
Inserted 10000 records. n: 436. Time: 20 
Inserted 10000 records. n: 437. Time: 20 
Inserted 10000 records. n: 438. Time: 22 
Inserted 10000 records. n: 439. Time: 22 
Inserted 10000 records. n: 440. Time: 26 
Inserted 10000 records. n: 441. Time: 24 
Inserted 10000 records. n: 442. Time: 69 
Inserted 10000 records. n: 443. Time: 133 
Inserted 10000 records. n: 444. Time: 17 
Inserted 10000 records. n: 445. Time: 18 
Inserted 10000 records. n: 446. Time: 20 
Inserted 10000 records. n: 447. Time: 19 
Inserted 10000 records. n: 448. Time: 19 
Inserted 10000 records. n: 449. Time: 20 
Inserted 10000 records. n: 450. Time: 18 
Inserted 10000 records. n: 451. Time: 18 
Inserted 10000 records. n: 452. Time: 19 
Inserted 10000 records. n: 453. Time: 19 
Inserted 10000 records. n: 454. Time: 19 
Inserted 10000 records. n: 455. Time: 22 
Inserted 10000 records. n: 456. Time: 33 
Inserted 10000 records. n: 457. Time: 25 
Inserted 10000 records. n: 458. Time: 23 
Inserted 10000 records. n: 459. Time: 24 
Inserted 10000 records. n: 460. Time: 134 
Inserted 10000 records. n: 461. Time: 31 
Inserted 10000 records. n: 462. Time: 21 
Inserted 10000 records. n: 463. Time: 21 
Inserted 10000 records. n: 464. Time: 21 
Inserted 10000 records. n: 465. Time: 20 
Inserted 10000 records. n: 466. Time: 25 
Inserted 10000 records. n: 467. Time: 26 
Inserted 10000 records. n: 468. Time: 24 
Inserted 10000 records. n: 469. Time: 25 
Inserted 10000 records. n: 470. Time: 31 
Inserted 10000 records. n: 471. Time: 28 
Inserted 10000 records. n: 472. Time: 31 
Inserted 10000 records. n: 473. Time: 32 
Inserted 10000 records. n: 474. Time: 33 
Inserted 10000 records. n: 475. Time: 31 
Inserted 10000 records. n: 476. Time: 40 
Inserted 10000 records. n: 477. Time: 120 
Inserted 10000 records. n: 478. Time: 23 
Inserted 10000 records. n: 479. Time: 30 
Inserted 10000 records. n: 480. Time: 33 
Inserted 10000 records. n: 481. Time: 28 
Inserted 10000 records. n: 482. Time: 26 
Inserted 10000 records. n: 483. Time: 31 
Inserted 10000 records. n: 484. Time: 22 
Inserted 10000 records. n: 485. Time: 26 
Inserted 10000 records. n: 486. Time: 25 
Inserted 10000 records. n: 487. Time: 22 
Inserted 10000 records. n: 488. Time: 38 
Inserted 10000 records. n: 489. Time: 24 
Inserted 10000 records. n: 490. Time: 27 
Inserted 10000 records. n: 491. Time: 42 
Inserted 10000 records. n: 492. Time: 39 
Inserted 10000 records. n: 493. Time: 97 
Inserted 10000 records. n: 494. Time: 59 
Inserted 10000 records. n: 495. Time: 28 
Inserted 10000 records. n: 496. Time: 24 
Inserted 10000 records. n: 497. Time: 25 
Inserted 10000 records. n: 498. Time: 24 
Inserted 10000 records. n: 499. Time: 37 
Inserted 10000 records. n: 500. Time: 24 
Inserted 10000 records. n: 501. Time: 30 
Inserted 10000 records. n: 502. Time: 31 
Inserted 10000 records. n: 503. Time: 38 
Inserted 10000 records. n: 504. Time: 65 
Inserted 10000 records. n: 505. Time: 61 
Inserted 10000 records. n: 506. Time: 47 
Inserted 10000 records. n: 507. Time: 31 
Inserted 10000 records. n: 508. Time: 38 
Inserted 10000 records. n: 509. Time: 29 
Inserted 10000 records. n: 510. Time: 31 
Inserted 10000 records. n: 511. Time: 30 
Inserted 10000 records. n: 512. Time: 33 
Inserted 10000 records. n: 513. Time: 30 
Inserted 10000 records. n: 514. Time: 35 
Inserted 10000 records. n: 515. Time: 35 
Inserted 10000 records. n: 516. Time: 33 
Inserted 10000 records. n: 517. Time: 33 
Inserted 10000 records. n: 518. Time: 30 
Inserted 10000 records. n: 519. Time: 46 
Inserted 10000 records. n: 520. Time: 75 
Inserted 10000 records. n: 521. Time: 30 
Inserted 10000 records. n: 522. Time: 36 
Inserted 10000 records. n: 523. Time: 28 
Inserted 10000 records. n: 524. Time: 29 
Inserted 10000 records. n: 525. Time: 32 
Inserted 10000 records. n: 526. Time: 34 
Inserted 10000 records. n: 527. Time: 43 
Inserted 10000 records. n: 528. Time: 38 
Inserted 10000 records. n: 529. Time: 37 
Inserted 10000 records. n: 530. Time: 29 
Inserted 10000 records. n: 531. Time: 29 
Inserted 10000 records. n: 532. Time: 30 
Inserted 10000 records. n: 533. Time: 33 
Inserted 10000 records. n: 534. Time: 39 
Inserted 10000 records. n: 535. Time: 55 
Inserted 10000 records. n: 536. Time: 73 
Inserted 10000 records. n: 537. Time: 50 
Inserted 10000 records. n: 538. Time: 54 
Inserted 10000 records. n: 539. Time: 31 
Inserted 10000 records. n: 540. Time: 38 
Inserted 10000 records. n: 541. Time: 50 
Inserted 10000 records. n: 542. Time: 40 
Inserted 10000 records. n: 543. Time: 21 
Inserted 10000 records. n: 544. Time: 38 
Inserted 10000 records. n: 545. Time: 34 
Inserted 10000 records. n: 546. Time: 41 
Inserted 10000 records. n: 547. Time: 34 
Inserted 10000 records. n: 548. Time: 31 
Inserted 10000 records. n: 549. Time: 37 
Inserted 10000 records. n: 550. Time: 58 
Inserted 10000 records. n: 551. Time: 42 
Inserted 10000 records. n: 552. Time: 36 
Inserted 10000 records. n: 553. Time: 38 
Inserted 10000 records. n: 554. Time: 55 
Inserted 10000 records. n: 555. Time: 39 
Inserted 10000 records. n: 556. Time: 45 
Inserted 10000 records. n: 557. Time: 36 
Inserted 10000 records. n: 558. Time: 48 
Inserted 10000 records. n: 559. Time: 45 
Inserted 10000 records. n: 560. Time: 36 
Inserted 10000 records. n: 561. Time: 32 
Inserted 10000 records. n: 562. Time: 36 
Inserted 10000 records. n: 563. Time: 46 
Inserted 10000 records. n: 564. Time: 51 
Inserted 10000 records. n: 565. Time: 49 
Inserted 10000 records. n: 566. Time: 47 
Inserted 10000 records. n: 567. Time: 49 
Inserted 10000 records. n: 568. Time: 45 
Inserted 10000 records. n: 569. Time: 41 
Inserted 10000 records. n: 570. Time: 39 
Inserted 10000 records. n: 571. Time: 43 
Inserted 10000 records. n: 572. Time: 68 
Inserted 10000 records. n: 573. Time: 41 
Inserted 10000 records. n: 574. Time: 33 
Inserted 10000 records. n: 575. Time: 54 
Inserted 10000 records. n: 576. Time: 39 
Inserted 10000 records. n: 577. Time: 45 
Inserted 10000 records. n: 578. Time: 50 
Inserted 10000 records. n: 579. Time: 34 
Inserted 10000 records. n: 580. Time: 36 
Inserted 10000 records. n: 581. Time: 47 
Inserted 10000 records. n: 582. Time: 59 
Inserted 10000 records. n: 583. Time: 40 
Inserted 10000 records. n: 584. Time: 37 
Inserted 10000 records. n: 585. Time: 35 
Inserted 10000 records. n: 586. Time: 59 
Inserted 10000 records. n: 587. Time: 62 
Inserted 10000 records. n: 588. Time: 26 
Inserted 10000 records. n: 589. Time: 54 
Inserted 10000 records. n: 590. Time: 54 
Inserted 10000 records. n: 591. Time: 47 
Inserted 10000 records. n: 592. Time: 53 
Inserted 10000 records. n: 593. Time: 41 
Inserted 10000 records. n: 594. Time: 45 
Inserted 10000 records. n: 595. Time: 39 
Inserted 10000 records. n: 596. Time: 45 
Inserted 10000 records. n: 597. Time: 55 
Inserted 10000 records. n: 598. Time: 50 
Inserted 10000 records. n: 599. Time: 47 
Inserted 10000 records. n: 600. Time: 39 
Inserted 10000 records. n: 601. Time: 48 
Inserted 10000 records. n: 602. Time: 43 
Inserted 10000 records. n: 603. Time: 57 
Inserted 10000 records. n: 604. Time: 38 
Inserted 10000 records. n: 605. Time: 42 
Inserted 10000 records. n: 606. Time: 44 
Inserted 10000 records. n: 607. Time: 55 
Inserted 10000 records. n: 608. Time: 46 
Inserted 10000 records. n: 609. Time: 49 
Inserted 10000 records. n: 610. Time: 44 
Inserted 10000 records. n: 611. Time: 43 
Inserted 10000 records. n: 612. Time: 43 
Inserted 10000 records. n: 613. Time: 48 
Inserted 10000 records. n: 614. Time: 36 
Inserted 10000 records. n: 615. Time: 55 
Inserted 10000 records. n: 616. Time: 58 
Inserted 10000 records. n: 617. Time: 39 

Y a-t-il un moyen de maintenir la performance à environ 9-10 secondes par 10000 enregistrements?

Répondre

0

Vos calculs se concentrent sur le côté clé (les clés vont-elles dans la RAM?). C'est important, mais après avoir atteint la taille de la base de données (qui est dominée par la valeur dans votre cas), les valeurs écrites sur le disque deviendront plus pertinentes pour la vitesse. Je crains que vous ne puissiez optimiser votre disque (aucun détail?) Dans une telle situation.

+0

Est-ce que KC garde les clés dans la mémoire? Ou est-ce qu'il les récupère du disque pour chaque écriture? Je ne vois aucune documentation mentionnant cela pour HashDB, bien qu'ils le mentionnent pour l'arbre B +. –

+0

L'utilisation totale de la RAM du système est d'environ 1,8 Go, sur laquelle le processus KC utilise environ 1,2 Go. Donc, je suppose que l'index n'est pas conservé dans la RAM. Est-ce vrai? Comment puis-je le forcer à conserver l'index dans la RAM? –

+0

Veuillez consulter la section "Mise en œuvre efficace de la base de données de hachage" dans le manuel. C'est une bonne idée d'ajuster la taille mmap, au moins dans tokyocabinet. – Sven

0

Avez-vous modifié aussi msiz?

Avez-vous essayé un paramètre bnum inférieur (quelque chose comme 1 ou 2 milliards)?

Quel est votre système de fichiers (ext4 a obtenu de meilleurs résultats que ext3 lors de mes tests)?

+0

msiz, comme je l'ai mentionné est de 2 Go. J'ai changé bnum à 1 milliard, et c'est toujours pareil. –

+0

Pouvez-vous nettoyer une certaine confusion en éditant? Dans votre question, vous avez dit que msiz est 30 Go. Si oui: Pour exclure les erreurs de débordement, pourriez-vous essayer quelque chose de sûr comme 1 ou 2 Go? – Sven

Questions connexes