Writing Avro records to HBase columns

I’ve been digging around recently to see how to store avro records in HBase without “exploding” values to single columns; this being a viable alternative since Hive 0.14 with it’s support for Avro queries in HBase columns.

What you basically need to do is to serialize your avro record to a byte array and put that in your column:

ByteArrayOutputStream baos = new ByteArrayOutputStream();
DatumWriter writer = new SpecificDatumWriter(SpecifcRecord.SCHEMA$);
DataFileWriter dfw = new DataFileWriter(writer);
PutRequest req = null;
try {
    dfw.create(SpecifcRecord.SCHEMA$, baos);
    dfw.append(currentEvent);
    dfw.close();
    req = new PutRequest(table, currentRowKey, colFam,
        "record".getBytes(), baos.toByteArray());
} catch (IOException e) {
    e.printStackTrace();
}

That’s basically it.

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s