mongodb - HBase or Mongo for an Analytics DB if already using Hadoop? -


i have hadoop cluster store tons of logs on run pig scripts calculating aggregated analytics. have mongo cluster store production data.

i've been put in position need lot of one-off analytics queries, or enable others them. these queries need use both production data , log data together, whatever go with, i'd have in 1 place. log data in json , 10x size of prod data. here pros/cons of mongo , hbase i'm seeing:

mongo pros/ hbase cons:

  1. since log data in json, can mongo pretty easily, , can in real time comes in through fluentd.
  2. most people work have experience writing mongo queries needing work prod data, getting analytics db on mongo simple use.
  3. i know less hbase mongo.
  4. no idea how easy/difficult data in json or mongo hbase. imagine isn't bad, don't see documentation.

hbase pros/mongo cons:

  1. my log data bigger prod data, storing in both hadoop , mongo way more expensive storing prod data in both hadoop , mongo.
  2. i can build hbase on top of running hadoop cluster , fit prod data in there without adding many machines. if went mongo, i'd need whole new mongo cluster.
  3. i use phoenix on top of hbase allow simple sql syntax accessing our data, i'm not sure how unwieldily multi-level document-based data.

i know little hbase currently, , wouldn't consider myself mongo expert, i'm missing lot.

so, missing, , right situation?

first of all, should use can handle. therefore, mongo db seems choice, when data in json format.

on other hand, used hbase quite while , read performance amazing although having lot of rows , don't know if there , fast integration of mongo db hadoop. hbase hadoop database, predestinated work hadoop together.

if logs indexed (in hbase rowkey):

producing_program_identifier, timestamp, ...

hbase work quite query pattern. if decide on hbase, use phoenix framwork, save time using familiar interfaces jdbc , sql-like queries. provides simple aggregation functions (count, avg, max, min) may sufficient.


Comments

Post a Comment

Popular posts from this blog

jquery - How can I dynamically add a browser tab? -

since using iframes house external pages are, accounts "not preferred method" , disallowed sites (flickr, one), want dynamically generate browser tabs. e.g., when user submits form, want dynamically generate tab on browser in response submitted. e.g., might want add new tab url like: http://www.bigsurgarrapata.com/contact how in jquery? you want form target attribute action attribute. http://www.w3schools.com/tags/att_form_target.asp browsers have ultimate control on result of target attribute open new tab value of _blank. <form action="contact_submit.php" target="_blank" method="post">
Read more

node.js - Getting the socket id,user id pair of a logged in user(s) -

i have web application done using node js , in it,i have authentication system requires users log in before can access pages. have 2 pages,the homepage , updates page. from read docs,a new socket id issued when user opens tab in browser. me being admin,i know socket id of logged in user , more so,the socket id,logged in user id pair logged in users. what method can use socket id,logged in user id pair make sure broadcast realtime info users online?.
Read more

keyboard - C++ GetAsyncKeyState alternative -

i using windows.h library's getasynckeystate function define key pushed on keyboard . here program returns int value of pushed key example. #include <iostream> #include <windows.h> using namespace std; int main (){ char i; for(i=8;i<190;i++){ if(getasynckeystate(i)== -32767){ cout<<int (i)<<endl; } } return 0; } but trying make opengl simple game , function appeared slow . there function not need cycle 180 times , if statement . thanks you're trying make opengl game, you're going use glfw. if that's case can away using glfwgetkey() such: if (glfwgetkey(glfw_key_up) == glfw_press) {...} if (glfwgetkey(glfw_key_down) == glfw_press) {...} that method nicely explained in tutorial 6 opengl-tutorial.org . should start first tutorial of beginners tutorials , later intermediate tutorials . if you're going make win32 window can listen wm_keydown or wm_keyup messages , virtua
Read more