Skip to content
This repository was archived by the owner on Sep 28, 2022. It is now read-only.
This repository was archived by the owner on Sep 28, 2022. It is now read-only.

Can I use pdk to ingest data from other database like Hive? #131

@BigRantLing

Description

@BigRantLing

Hi guys:
I'm a freshman of Pilosa and go. I read some docs in your web site, and I am not clear with Pilosa and it's data model so far.
For example, some records of relational database like below:

name age
A 18
B 19

It will be transformed to bitmap index:

Field name

ID 1 2
A 1 1 0
B 2 0 1

Field age (BSI)

age 18 19
comp 0 1 0
comp 2 0 0
comp 3 1 1
comp 4 1 1
comp 5 0 0
not_null 1 1

Is that right?

If I understand the above correctly. I have the following confusion:

  1. Pilosa store row and column ids in field, how can I transform the string values to int ids while ingestion. For example Field name, how can I transform A to Row id 1 while ingestion. The map A to 1 need to be maintained in other server of user's system or pilosa will do it automatically?If it's latter, can I get the value A while query Row(name=1)?
  2. If I want to ingest data from hive, what should I do ? Is there any DML like batch query or it need users to set Rows of field one by one?

Excepting to hear from you guys and thanks for helping me to use and understand Pilosa better !

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions