10/08/2025
Random News

EzoitAI

Apache Hive

Quản trị19210 mins

Giới Thiệu

Apache Hive là một hệ thống lưu trữ dữ liệu được xây dựng trên nền tảng của Hadoop, hỗ trợ phân tích các tập dữ liệu lớn được lưu trữ trong các hệ thống tệp phân tán của Hadoop (HDFS) hoặc S3 bằng cách sử dụng các truy vấn tương tự SQL.

Nó lưu trữ siêu dữ liệu (metadata) của schema trong một cơ sở dữ liệu (như Apache Derby, MySQL, Postgres hoặc MariaDB) và xử lý dữ liệu từ lưu trữ kết nối (như HDFS hoặc S3).

Nó cung cấp ngôn ngữ truy vấn giống SQL, HiveQL.

Hive Metastore

Hive metastore ghi lại tất cả thông tin cấu trúc (metadata) của một bảng Hive và các phân vùng của nó trong kho lưu trữ:

  • Thông tin cột và định nghĩa kiểu dữ liệu
  • Các serializer và de-serializer cần thiết để đọc và ghi dữ liệu
  • Vị trí của dữ liệu liên quan trong HDFS hoặc S3

Các siêu dữ liệu trong Hive metastore được lưu trữ trong một hệ quản trị cơ sở dữ liệu mã nguồn mở là Apache Derby (MySQL/Postgres/MariaDB cũng có thể được sử dụng).

Hive metastore có thể có nhiều schema (cơ sở dữ liệu), và mỗi schema (cơ sở dữ liệu) có thể chứa nhiều siêu dữ liệu bảng Hive.

Bảng Hive

Loại bảng Hive

Hive hỗ trợ hai loại bảng: Bảng Quản lý (Bảng Nội bộ) & Bảng Ngoại bộ

Bảng Quản lý (Bảng Nội bộ)

Bảng Quản lý: còn được gọi là bảng nội bộ, các bảng này quản lý cả dữ liệu và siêu dữ liệu trong thư mục kho lưu trữ mặc định của Hive (được chỉ định bởi hive.metastore.warehouse.dir).

Bảng Ngoại bộ

Bảng Ngoại bộ: các bảng này tham chiếu đến các tệp dữ liệu được lưu trữ bên ngoài Hive, thường là trong HDFS hoặc S3.

Vì các tệp dữ liệu không được quản lý bởi Hive, nên việc thay đổi hoặc xóa một bảng Hive ngoại bộ không xóa dữ liệu ở dạng dưới lying. Bạn có thể cần xóa dữ liệu từ vị trí HDSF hoặc S3 phía dưới.

Cách Tạo Bảng Hive

Bước 1: Tạo SparkSession với cấu hình Hive

pythonSao chép mãspark = SparkSession.builder \
    .appName("Hive Table Example") \
    .config("spark.sql.warehouse.dir", "/user/hive/warehouse") \
    .enableHiveSupport() \
    .getOrCreate()

Bước 2: Có hai phương pháp để tạo bảng Hive

Phương pháp 1: Tạo bảng Hive bằng cách sử dụng API DataFrame với write.saveAsTable

pythonSao chép mã# Bảng Quản lý (Bảng Nội bộ)
df.write \
  .mode("overwrite") \
  .option("database", "your_database_name") \
  .partitionBy("partition_col1", "partition_col2") \
  .saveAsTable("hive_table_name")

# Bảng Ngoại bộ
df.write \
  .mode("overwrite") \
  .option("database", "your_database_name") \
  .option("path", "s3a://your_bucket_name/your_folder_path/your_external_table_name") \
  .partitionBy("partition_col1", "partition_col2") \
  .format("parquet") \
  .saveAsTable("hive_table_name")

Phương pháp 2: Tạo Bảng Hive bằng cách sử dụng Truy vấn SQL với spark.sql

pythonSao chép mã# Bảng Quản lý (Bảng Nội bộ)
spark.sql("""
    CREATE TABLE IF NOT EXISTS your_database_name.your_managed_table_name (
        id INT, name STRING, age INT
    )
    PARTITIONED BY (country STRING, city STRING)
    USING PARQUET
""")

# Bảng Ngoại bộ
spark.sql("""
    CREATE TABLE IF NOT EXISTS your_database_name.your_external_table_name (
        id INT, name STRING, age INT
    )
    PARTITIONED BY (country STRING, city STRING)
    STORED AS PARQUET
    LOCATION 'hdfs://your_hdfs_path/your_external_table_name/'
""")

Cách chèn dữ liệu vào Bảng Hive

Phương pháp 1: Ghi trực tiếp thông qua write.saveAsTable vào Bảng Hive

pythonSao chép mãdf.write \
  .mode("overwrite") \
  .partitionBy("partition_col1", "partition_col2") \
  .format("parquet") \
  .saveAsTable("your_database_name.your_hive_table_name")

Phương pháp 2: Ghi vào vị trí lưu trữ dưới lying & cập nhật lại siêu dữ liệu Hive

pythonSao chép mãtable_path = "s3a://your_bucket_name/your_folder_path/your_hive_table_name"

df.write \
  .mode("overwrite") \
  .partitionBy("partition_col1", "partition_col2") \
  .parquet(table_path)

spark.sql("REFRESH TABLE your_database_name.your_hive_table_name")

spark.sql("MSCK REPAIR your_database_name.your_hive_table_name")

Cách Xem Chi Tiết Bảng Hive

Lấy thông tin chi tiết về bảng

pythonSao chép mãtable_metadata = spark.sql("DESCRIBE FORMATTED db.table_name")
table_metadata.show(truncate=False)

Lấy thông tin về phân vùng của bảng

pythonSao chép mãpartitions = spark.sql("SHOW PARTITIONS db.table_name")
partitions.show(truncate=False)

Đây là một cách sơ bộ để làm việc với Hive trong Spark. Có nhiều khía cạnh và tính năng hơn mà bạn có thể khám phá!

192 thoughts on “Apache Hive”

  102. В этой информационной статье вы найдете интересное содержание, которое поможет вам расширить свои знания. Мы предлагаем увлекательный подход и уникальные взгляды на обсуждаемые темы, побуждая пользователей к активному мышлению и критическому анализу!
    Получить дополнительные сведения – https://medalkoblog.ru/

    Bình luận

  140. ¡Saludos, estrategas del desafío !
    Casino sin licencia EspaГ±a sin enviar documentaciГіn – п»їcasinossinlicenciaenespana.es casinos sin licencia
    ¡Que vivas movimientos brillantes !

    Bình luận

  147. ¡Saludos, aficionados a los desafíos!
    ВїQuГ© juegos ofrece un casino online extranjero? – п»їhttps://casinoextranjerosenespana.es/ casinoextranjerosenespana.es
    ¡Que disfrutes de giros espectaculares !

    Bình luận

  150. ¡Bienvenidos, exploradores de la fortuna !
    Casino fuera de EspaГ±a para jugadores internacionales – п»їhttps://casinoporfuera.guru/ casinos online fuera de espaГ±a
    ¡Que disfrutes de maravillosas botes impresionantes!

    Bình luận

  152. ¡Bienvenidos, exploradores de oportunidades !
    casinofueraespanol con promociones por cada recarga – п»їhttps://casinofueraespanol.xyz/ casinofueraespanol
    ¡Que vivas increíbles botes deslumbrantes!

    Bình luận

  158. Hello navigators of purification !
    Best Purifier for Smoke – Powerful Yet Quiet – п»їhttps://bestairpurifierforcigarettesmoke.guru/ smoke purifier
    May you experience remarkable immaculate environments !

    Bình luận

  166. Looking for reliable appliance repair near you? Our expert technicians offer fast, affordable, and efficient repairs for all major appliances. Whether it’s a fridge, washer, or oven, we provide trusted service with a satisfaction guarantee. Get your appliances fixed today—contact us for same-day appointments!

    Bình luận

  170. Hello envoys of vitality !
    Switching to the best air filters for pets reduces the number of airborne irritants your family breathes each day. Top rated air purifiers for pets use multi-stage systems to clean the air more effectively than basic filters. Selecting the best air purifier for pet allergies ensures long-term relief and better breathing.
    Installing an air purifier for dog smell neutralizes odors using activated carbon filters. These units are perfect for entryways or near pet crates air purifier for petsThey eliminate that “wet dog” aroma quickly and effectively.
    Best Air Purifier for Pet Allergies with Fast Results – п»їhttps://www.youtube.com/watch?v=dPE254fvKgQ
    May you enjoy remarkable filtered environments !

    Bình luận

  173. As the rocket flies deep into space, you can claim decent wins, including multipliers. In this Space XY Crash Game review, we will look extensively into this title powered by BGaming, and how you can start playing for real money. Avia Rush is straightforward, sharp-looking, and easy to jump into. If you like games that are all about timing and risk management, this is worth a look. This game, however, doesn’t require 3D printing at all. Yes, there are 3D models out there that you could make for a more robust game of tic-tac-toe, but you could also just draw the pound (hashtag) symbol on a piece of paper and dive into a game that way. Do not like to raise the balance alone, or want to discuss actions with other players – we have created an in-game chat for you: it contains all users currently playing Space XY. I wonder who raised the most – this is displayed in the chat. Try to get yourself into the place of those who snatched a higher coefficient.
    https://hoodiechicken.com/mobile-vs-desktop-multiplier-speed-in-jetx-measured-results/
    CornerProBet is the ultimate football analysis website with all the football statistics you need for your football betting. Over Under Goals, Over Under Corners, Both Teams to Score (BTTS) are some of the soccer stats you can find. Gal Sport Betting provides customer support for users facing issues or requiring assistance with their betting accounts. In 2018, the Supreme Court gave U.S. states permission to legalize sports betting if they wish to do so. It is still fully illegal in 8 states, including California, Utah, and Texas. World Trade Centre, Curacao Business Centre 4th Floor Suite BC.IV.09 Piscadera Bay Willemstad Curacao Goalbet won’t charge any fees for withdrawals as long as you request a minimum of €30. Goalbet withdrawals are subject to a pending period of 24–48 hours. This is the time it takes for Goalbet to verify and process the withdrawal request. Once the withdrawal is approved, eWallet users should have their funds within 24 hours while bank transfers and card withdrawals should get to you in 3–5 business days. Refer to the following table for withdrawal methods, times, and minimum and maximum withdrawal amounts.

    Bình luận

  175. Supercharged reels рџЋЈEveryone’s favourite fisherman has joined our elite 1000 series as he sets sail for wins of up to 20,000x ✨ 18+ t.co jIl6UrUvsP#PragmaticPlay #YourFavouriteEveryTime #BigBassBonanza1000 #slots pic.twitter 4FRNrsrOKF Información importante Con una configuración tradicional de 5 rodillos por 3 filas, la tragaperras Big Bass: Keeping it Reel debuta en el mercado chileno y en otras jurisdicciones latinoamericanas. El juego también cuenta con 10 líneas de pago y un rango de apuesta variable. En algunos casos, el límite oscila entre 0,10€ y 50€, mientras que en otras circunstancias, varía entre 0,10€ y 250€ o su equivalente en pesos chilenos. Con un poco de suerte, los jugadores pueden conseguir grandes premios en estos slots. En Bigger Bass Bonanza™ serás un pescador en busca de premios en efectivo, así que debes reunir símbolos de cañas, barcos y carnadas para poder generar pagos proporcionales a tus apuestas.
    https://wellsuctingham1975.bearsfanteamshop.com/informacion
    Ultra Hold y Spin tiene dos símbolos de dinero: monedas de plata y oro. Una vez que aterrices al menos tres de ellos en el segundo carrete, puedes obtener de 1x a 20x tu apuesta inicial. La tragamonedas Big Bass Secrets of the Golden Lake, como sus predecesoras de la serie Big Bass, ofrece al jugador una mezcla de buena mecánica de juego, tema original y funciones de bonificación atractivas. Su capacidad para combinar un tema y otro medieval lo hace único, proporcionando una grata experiencia a los jugadores. En cuanto a las opciones de apuesta, Big Bass Bonanza ofrece diferentes tamaños de apuesta. Así, puedes decidir si quieres apostar 0,10 o 250 créditos por giro. Sí. Ultra Burn está disponible para jugar sin registrarse sin dinero real. Pragmatic Play y Reel Kingdom nos presenta un juego tragamonedas donde se combinan aguas misteriosas con la leyenda medieval del Rey Arturo. Hablamos de Big Bass Secrets of the Golden Lake y podemos disfrutar de esta tragamonedas desde el 8 de abril de 2024. Con el legendario pescador de Big Bass podremos pescar en el lago dorado para conseguir grandes premios, incluso lograr un premio 5.000x la apuesta. En Chilebets te contamos todos los detalles de esta tragamonedas.

    Bình luận

  178. Часто бывает так: ставишь ставку, ждёшь — и ничего. Но здесь всё иначе. С первой прокрутки почувствовал азарт, с третьей — сорвал неплохой выигрыш. А дальше — больше. Интуитивно понятный интерфейс, быстрые выплаты, бонусы почти на каждом шагу. https://vodka-registration.site стал моим ежедневным порталом в мир адреналина и шанса. То, что зеркало открывается само, — это плюс, особенно в условиях блокировок. А мобильное приложение вообще откровение: всё быстро, чётко, удобно. Радует, что есть акции, а не просто «бонус за депозит». Тут действительно борются за игрока, и это чувствуется.

    Bình luận

  182. To begin your journey, visit Roobet and locate Mission Uncrossable in the game library. If you’re new, create an account and deposit funds using Bitcoin, Ethereum, or Litecoin. Mission Uncrossable offers a unique experience with its combination of high volatility and substantial jackpots. By understanding the game’s mechanics, employing effective strategies, avoiding common pitfalls, and maintaining a positive mindset, players can enhance their chances of success. Remember that responsible gambling is essential, so always set limits on your spending and enjoy the game within your means. High-volatility slots are designed for players seeking larger rewards but with less frequent wins. These slots typically have higher variance, meaning that while you may experience fewer immediate payouts, the potential for a significant jackpot is greater. Mission Uncrossable exemplifies this characteristic, missionuncrossable-demo offering players the chance to win big without relying on consistent small wins.
    https://seomotionz.com/member.php?action=profile&uid=78307
    Connect with us The multi-colored lollipop is a scatter symbol that triggers 10 free spins when it lands in at least 3 places. Any 3 lollipops during a free game retrigger the round, and tumbling reels are still active in these spins. During the bonus spins, a rainbow-style Christmas bauble can appear, revealing a win multiplier and sticking in place for any remaining games. At the end of the feature, the values of any baubles are added up and applied to the combined wins from the free games. Sweet Bonanza is not a progressive jackpot slot but offers a maximum win of 21,100 times your total stake, making it a highly appealing slot for high-roller players. If you like your gameplay with just a touch of sweetness, then this Sweet Bonanza slot could be your next new favourite, just don’t tell your dentist!

    Bình luận

  184. GODZINY URZĘDOWANIA ODPOWIEDZIALNA GRA: W aviatorgame.net priorytetowo traktujemy odpowiedzialną grę, ponieważ wierzymy, że gra w kasynie online powinna być zabawnym i przyjemnym doświadczeniem. Jeśli kiedykolwiek poczujesz się niekomfortowo z powodu utraty pieniędzy, koniecznie zrób sobie przerwę i cofnij się na chwilę. Zapewniamy również różne strategie, które pomogą ci zarządzać grą w zdrowy sposób. Twoja przyjemność i dobre samopoczucie są dla nas najważniejsze! Powered by Tannbit Jesteśmy gotowi oświadczyć, że dostawca oprogramowania miał na uwadze zarówno początkujących, jak i doświadczonych graczy podczas projektowania interfejsu Aviator online. Po pierwszym uruchomieniu zobaczysz, że wszystko jest na swoim miejscu i wygodnie rozmieszczone na stronie. Obszar zakładów, w którym można ustawić wysokość stawki, a także wybrać jeden lub dwa zakłady i kilka innych opcji, znajduje się poniżej głównego ekranu z samolotem. Lewa strona poświęcona jest historii zakładów, podczas gdy w prawej części znajduje się kilka opcji, w tym ustawienia gry i Prawdopodobnie uczciwe. Aviator Gra pozwala nawet graczom oglądać poprzednie kursy powyżej ekranu głównego.
    https://luhachsolutions.com/jak-skutecznie-zalogowac-sie-do-polskiej-wersji-serwisu-vulkan-vegas/
    Pragmatic Play słynie z wysokiej jakości automatów, gier stołowych i rozrywki z krupierem online, które sprawiają, że gra jest ekscytująca i bezpieczna. Pragmatic dokłada wszelkich starań, aby każdy tworzony produkt był jak najbardziej ekscytujący. Ponadto dostawcy Lemon Casino online i slotów wideo mogą organizować turnieje dla swoich gier. W ten sposób możesz zarobić bonusowe obroty, a nawet gotówkę. Podczas gdy turnieje na slotach mogą być świetną zabawą, jest kilka rzeczy, które musisz wiedzieć przed przystąpieniem do nich. Woodlanders to slot online od Betsoft, który oferuje oszałamiające efekty audiowizualne, Stacked Mystery Symbols, Multiplier Wilds i Darmowe Spiny z RTP 96,06%. Software: Pragmatic Play Pragmatic Play, wiodący dostawca treści dla branży iGaming, rozpoczął nowe przedsięwzięcie z Dinastia, w ramach którego cały katalog dostawcy zostanie udostępniony na platformie operatora. 

    Bình luận

  185. Here, you get a clean design, fast games, and features that work. From Megaways slots to blackjack tables with real dealers. No app necessary. No hidden settings. No wandering through tabs to find favourites. Everything you need. Nothing you don’t. Licenced and reputable online casinos do not rig their games. Firstly, they risk losing their licence to operate if they rig their games. Secondly, casino games have a built-in advantage in favour of the casino (the house edge) that ensures their profits in the long run, so they have no need to rig casino games.  Gambling can be harmful if not controlled and may lead to addiction! Use our online tools and play responsibly. Buffalo King Untamed Megaways Not here. Every slot game, table, and payout system is built to load fast and play sharp. No app, no delays. Simply smooth access to your favourite casino games wherever you are.
    https://badninja9.com/what-indian-gamers-say-about-aviatrix.html
    Some games will offer a no-deposit bonus offering coins or credits, but remember, free slots are just for fun. So, whilst you may miss the thrill of a real money prize or big cash bonuses, you will however benefit from the fact that you can’t lose real money either. Bodog stands as a top online casino which players have trusted since 1994. Our site is one of Canada’s top gaming destinations with more than 1,000 real money casino games and features new slots and classic table games as well as real live dealer experiences. For those eager to jump straight into the action, Almighty Buffalo Megaways is one of White Hat Studio’s slots with a bonus buy feature. “Buffalo Bonus Bet” lets you buy free spins for 100x your bet, instantly triggering a minimum of four scatter symbols and launching the free spins round. Choose your free spins strategy wisely — with options ranging from 15 spins at a 1x multiplier to a mysterious mix that could yield anything from spectacular to modest outcomes.

    Bình luận

  187. ¡Saludos a todos los visitantes del casino !
    Casasdeapuestassindni.guru ofrece acceso a plataformas anГіnimas. Apuestas deportivas sin dni estГЎn disponibles sin registro. casas de apuestas sin dni Casas de apuestas SIN verificaciГіn aceptan criptomonedas y tarjetas virtuales.
    Casasdeapuestassindni ofrece plataformas rГЎpidas y seguras. Casa de apuestas sin dni elimina esperas innecesarias. Casas de apuestas SIN registro dni permiten jugar de forma inmediata.
    Usa casasdeapuestassindni sin entregar datos – п»їhttps://casasdeapuestassindni.guru/
    ¡Que goces de increíbles premios !

    Bình luận

  191. Zo, dat gezegd hebbende….. dit is een prachtige peper. Helaas nog niet helemaal stabiel. Dat is ook een proces waar nu nog aan gewerkt wordt. De eerste peper met streepjes is in Noorwegen gevonden en sindsdien heeft ze een lange weg langs andere kwekers en zelfs de universiteit van New Mexico afgelegd om te achterhalen waar de streepjes vandaan komen en hoe ze kunnen worden behouden. Kijk vooral even in dit document voor meer informatie daarover: Fatalii.net Erkend door de Belgische Kansspelcommissie Als je in Nederland op zoek bent naar de opwindende Sugar Rush gokkast van Pragmatic Play, is het belangrijk om te weten dat je deze alleen kunt spelen bij een online casino dat beschikt over een geldige licentie van de Kansspelautoriteit. Dit houdt in dat alleen casino’s die in het bezit zijn van deze vergunning de Sugar Rush gokkast mogen aanbieden wanneer er om geld wordt gespeeld.
    https://astraonline.com.br/sugar-rush-casino-analyse-rtp-consistentie-en-winstructuur/
    De oogverblindende graphics en de levendige soundtrack zorgen ervoor dat de Sugar Rush slot boven de concurrentie uitsteekt. Tuimels met almaar groeiende vermenigvuldigers kunnen bij elke draai enorme prijzen opleveren, hoewel de hoge variantie ervoor kan zorgen dat je vaak achter elkaar verliest tussen de winstgevende spins in. Wie heeft Sugar Rush ontwikkeld? Als je in Nederland op zoek bent naar de opwindende Sugar Rush gokkast van Pragmatic Play, is het belangrijk om te weten dat je deze alleen kunt spelen bij een online casino dat beschikt over een geldige licentie van de Kansspelautoriteit. Dit houdt in dat alleen casino’s die in het bezit zijn van deze vergunning de Sugar Rush gokkast mogen aanbieden wanneer er om geld wordt gespeeld. In Sugar Rush verdwijnen winnende symbolen en maken plaats voor nieuwe van bovenaf, dankzij de Tumble feature. Deze cyclus gaat door totdat er geen nieuwe winnende combinaties meer worden gevormd. Met een RTP van 96,5% en een hoge volatiliteit, kunnen grote overwinningen plaatsvinden, maar ze komen niet zo vaak voor als bij slots met lagere variantie.

    Bình luận

Để lại một bình luận

Email của bạn sẽ không được hiển thị công khai. Các trường bắt buộc được đánh dấu *

Related News