hive - What is the syntax to create an external table partitioned on an hbase column? -


i have table in hbase i'd represent external table in hive

so far i've been using:

create external table events(key string, day int, source string, ip string) stored 'org.apache.hadoop.hive.hbase.hbasestoragehandler' serdeproperties ("hbase.columns.mapping"=":key,c:date#b,c:source,c:ipaddress") tblproperties ("hbase.table.name" = "eventtable");

however queries aren't balanced across mappers, i'm trying partition on ip address:

create external table events(key string, source string) partitioned (ip string) stored 'org.apache.hadoop.hive.hbase.hbasestoragehandler' serdeproperties ("hbase.columns.mapping"=":key,c:date#b,c:source,c:ipaddress") tblproperties ("hbase.table.name" = "eventtable");

but receive error improper column mappings:

failed: error in metadata: java.lang.runtimeexception:     metaexception(message:org.apache.hadoop.hive.serde2.serdeexception   org.apache.hadoop.hive.hbase.hbaseserde: columns has 2 elements while hbase.columns.mapping has 3 elements (counting key if implicit)) failed: execution error, return code 1 org.apache.hadoop.hive.ql.exec.ddltask

i've been looking around can't find documentation indicates how map between hbase column , hive partitioning column

i think can't partition external table easily, when underlying storage hbase.

hive partition strategy build on way data specific partition stored in separate folder ("or other storage"). because of partitioning hbase (if exists) require usage of more tables or usage of hbase versions.

i think post give better understanding of partitioning http://blog.zhengdong.me/2012/02/22/hive-external-table-with-partitions

and on place https://cwiki.apache.org/hive/hbaseintegration.html can find partitioning in hbase left future.

if want have partitions recommend loading data hbase/hive hdfs/hive table, depends on use cases.

regards, dino


Comments

Post a Comment

Popular posts from this blog

Change php variable from jquery value using ajax (same page) -

what i'm trying this: i've php global variable $currentid , simple jquery script check if there "id" (page) in hash of url , if yes wanted give value $currentid . i've been reading lot , notice shall use ajax i'm not managing it. my code is: <script type="text/javascript"> $(document).ready(function(){ if (parent.location.hash != '' ) { var thecurrentid = parent.location.hash.split('#')[1]; $.ajax({ url: 'my.php', //current page type: 'post', data: {theid: thecurrentid }, datatype: 'json', success: function () { } }); } // code here }); </script> could me please? thank in advance. php server side language, meaning once has sent data browser, php variables , data cannot changed... however, using ajax, can send request php page , change data in dom based on result. consider th...
Read more

Pull out data related to my apps from Android Play Store and iOS App Store -

i want pull out data related apps (ratings, reviews, etc) google play store , apple app store ? how can accomplish ? i have found open source apis google mentioned not reliable. also there paid apis (e.g. appfigures) available. has idea how able pull data these play stores ?? i looking direction on how can accomplish this. can suggest ? we (appfigures) have been working on developing infrastructure give ability reviews , bunch of other data points major app stores several years :] we make available via our api though may want check out.
Read more

How can I fetch data from a web server in an android application? -

i want retrieve data web server in android application, , don't know begin. should use web services? i recommend these tutorials: connect android php , mysql , json in android , php , mysqli i used these tutorials , managed trying working without difficulty. between them describe each step in how attempting @ each stage, android application, database , web server side , has information included can process , use received information the thing add connect android php , mysql tutorial makes use of mysql_ in php deprecated. better use mysqli why included third link. the basic outline of want this: 1) in android app make request server php script using class this: import java.io.bufferedreader; import java.io.ioexception; import java.io.inputstream; import java.io.inputstreamreader; import java.io.unsupportedencodingexception; import java.util.list; import org.apache.http.httpentity; import org.apache.http.httpresponse; import org.apache.http.namevaluepair...
Read more