Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ingestion Failure: "Failed to find a registered source for type hive-metastore" in Ingestion #11389

Open
vegaskyo opened this issue Sep 17, 2024 · 0 comments
Labels
bug Bug report

Comments

@vegaskyo
Copy link

Description:
I encountered an issue while trying to configure DataHub's ingestion to connect to a Hive Metastore (via PostgreSQL) using the Web UI.

Issue Details:

  • Deployment Environment: Kubernetes
  • Error Message: "Failed to find a registered source for type hive-metastore" in the ingestion UI.
  • Verification Attempts:
    • I checked the installed plugins by running datahub check plugins --verbose inside the DataHub pod.
    • It only lists hive and presto-on-hive but no plugin for hive-metastore.

Steps to Reproduce:

  1. Configured Hive Metastore ingestion through the Web UI -> Other

  2. Connected via PostgreSQL in the ingestion section. I follow the doc:
    `source:
    type: hive-metastore
    config:
    host_port: localhost:5432
    database: metastore

    schema_pattern:
    allow:
    - "^public"
    username: user
    password: pass # optional

    scheme: 'postgresql+psycopg2' # set this if metastore db is using postgres`

  3. Ran the ingestion and got the error message: "Failed to find a registered source for type hive-metastore."

  4. Verified the plugins in the DataHub pod, but hive-metastore was missing.

Expected Behavior:
As per the DataHub documentation, there should be support for hive-metastore. I expected the ingestion source to be recognized and executed successfully.

Current Behavior:
The UI and CLI fail to recognize hive-metastore as a valid source for ingestion.

Additional Information:

  • DataHub Version: v0.14.0.2

Is the hive-metastore plugin available or supported? If so, could it be a misconfiguration on my end, or is there a discrepancy between the documentation and the actual available sources?

@vegaskyo vegaskyo added the bug Bug report label Sep 17, 2024
@vegaskyo vegaskyo changed the title Ingestion Failure: "Failed to find a registered source for type hive-metastore" in Hive-Metastore Ingestion Ingestion Failure: "Failed to find a registered source for type hive-metastore" in Ingestion Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug report
Projects
None yet
Development

No branches or pull requests

1 participant