Spark Session

By /

Giới thiệu

SparkSession là gì?

Bước đầu tiên trong bất kỳ Ứng dụng Spark nào là tạo một SparkSession. SparkSession hoạt động như quá trình điều khiển để kiểm soát ứng dụng Spark. Khi bạn khởi đầu Spark trong chế độ tương tác, một SparkSession được tạo một cách ngầm định để quản lý Ứng dụng Spark. Tuy nhiên, khi khởi động nó thông qua một ứng dụng độc lập, bạn cần tự tạo đối tượng SparkSession trong mã ứng dụng của bạn.

SparkSession so với SparkContext

Trong Spark 1.X, có hai ngữ cảnh hiệu quả: SparkContext và SQLContext. SparkContext tập trung vào kiểm soát tinh tế hơn của các trừng tượng trung tâm của Spark, trong khi SQLContext xử lý các công cụ cấp cao hơn như Spark SQL. Tuy nhiên, trong Spark 2.X, cộng đồng đã hợp nhất hai API này thành SparkSession tập trung để quản lý tốt hơn và ngăn chặn xung đột ngữ cảnh. Mặc dù SparkContext và SQLContext vẫn có sẵn trong SparkSession, nhưng hiếm khi được sử dụng trực tiếp.

Tạo SparkSession

Một SparkSession là điểm vào tất cả các chức năng trong Spark và là bắt buộc nếu bạn muốn xây dựng một DataFrame trong PySpark.

pythonSao chép mãspark = SparkSession \
        .builder \
        .appName("SparkExample") \
        .enableHiveSupport() \  # Để làm việc với Hive metastore.
        .config("spark.memory.offHeap.enabled","true") \ # Bộ nhớ cache dữ liệu trong bộ nhớ off-heap để tránh lưu trữ nó trực tiếp trên đĩa.
        .config("spark.memory.offHeap.size","10g") \
        .getOrCreate() # Tạo Spark session với các cấu hình đã chỉ định.

Một ví dụ khác với nhiều cấu hình hơn:

pythonSao chép mãdef create_spark_session():
    '''
    Hàm để tạo một Spark session.
    Output:
        SparkSession
    '''
    try:
        spark = SparkSession \
             .builder \
             .appName("SparkExample") \
             .enableHiveSupport() \ # Để làm việc với Hive metastore.
             .config("spark.driver.memory", "9g") \
             .config("spark.executor.cores", "2") \
             .config("spark.executor.memory", "9g") \
             .config("spark.yarn.queue", "root.tnm.ada_analytics_tnm") \ # Cần cập nhật với tên Queue YARN.

        custom_spark_config = {
            "spark.driver.memory": "9g",
            "spark.executor.cores": "2",
            "spark.executor.memory": "9g",
            "spark.yarn.queue": "root.tnm.ada_analytics_tnm", # Cần cập nhật với tên Queue YARN.
         }

        for key, value in custom_spark_config.items():
            spark = spark.config(key, value)

        spark = spark.getOrCreate() # Tạo Spark session với các cấu hình đã chỉ định.

        return spark

    except RuntimeError as e:
        print('Lỗi khi tạo Spark context')
        print(e)

Hàm này tạo một Spark session với các cấu hình đã chỉ định, hỗ trợ kiểm soát tinh tế hơn đối với ứng dụng Spark.

91 thoughts on “Spark Session”

  3. LuckyNiki_gt2

    has anyone here tried luckyniki before? im thinking about signing up but want to know if its actually legit or if its just another sketchy site lol. A freind told me about it but i want teh real opinions from people who’ve actually used it. Should i give it a go or nah? luckyniki app

    Reply
  6. Mega777_iv7

    Hello i want share about mega777 is very good website for me. I use this site many times and is very easy for understand the system. The people who work in support is very helpfull when I have questions about account. I think is good choice for peoples who is new and want to learn how platform work.

    Reply
  12. 26a

    26a… Feels fresh, a lot of new markets available compared to other sites that I have used in the past. All the information you need here: 26a

    Reply
  19. phukriKi

    Создание рефератов теперь стала проще благодаря актуальным нейросетям, которые позволяют намного быстрее находить материалы, делать план работы, выстраивать структуру текста и избавляться от части рутинной работы. Если нужен понятный список таких сервисов, можно посмотреть рейтинг нейросетей для рефератов на сайте http://neyroseti-dlya-referatov.ru/, где собраны популярные решения для учебы, сравнение их функций и полезные рекомендации по выбору.

    Reply
  24. CrickexCx_cr4

    If your looking for something reliable and straightforward, you should definately check out cx crickex—they’ve got a clean setup and decent variety. No hidden surprises, which is refreshing honestly.

    Reply
  25. phukehKi

    Создание рефератов сегодня стала намного проще благодаря новым нейросетям, которые позволяют намного быстрее находить материалы, формировать план текста, выстраивать структуру текста и тратить меньше времени на однотипные действия. Если хочется быстро сравнить такие сервисы, можно посмотреть рейтинг нейросетей для рефератов на сайте http://neyroseti-dlya-referatov.ru/, где представлены популярные сервисы для учебы, сравнение возможностей сервисов и рекомендации для выбора подходящего варианта.

    Reply
  26. phukllKi

    Работа над рефератами заметно упростилась благодаря умным нейросетям, которые дают возможность намного быстрее находить материалы, быстро набрасывать план, улучшать структуру текста и избавляться от части рутинной работы. Если нужен удобный обзор таких сервисов, можно изучить рейтинг нейросетей для рефератов на сайте http://neyroseti-dlya-referatov.ru/, где собраны популярные решения для учебы, сравнение их функций и понятные рекомендации по подбору сервиса.

    Reply
  28. phuknlKi

    Подготовка рефератов сегодня стала намного проще благодаря умным нейросетям, которые дают возможность намного быстрее находить материалы, формировать план текста, улучшать подачу материала и избавляться от части рутинной работы. Если нужен понятный список таких сервисов, можно изучить рейтинг нейросетей для рефератов на сайте http://neyroseti-dlya-referatov.ru/, где есть востребованные инструменты для учебных задач, сравнение их функций и полезные советы по выбору.

    Reply
  29. phukdrKi

    Написание рефератов заметно упростилась благодаря современным нейросетям, которые помогают оперативно находить нужную информацию, делать план работы, улучшать подачу материала и тратить меньше времени на однотипные действия. Если нужен понятный список таких сервисов, можно изучить рейтинг нейросетей для рефератов на сайте http://neyroseti-dlya-referatov.ru/, где есть востребованные инструменты для учебных задач, сопоставление их возможностей и полезные рекомендации по выбору.

    Reply
  30. LuckyStar_de9

    Been using Lucky Star for like 2 weeks now and the interface is pretty intuitive, loads fast even on my phone which I apreciate, though the customer support took forever to respond when I had a question about my account settings. Overall not bad though. lucky star

    Reply
  31. TeenPatti_ug5

    Good day everyone, I wanted to share my thoughts on this particular application which I have been using for some weeks now. The performance has been quite satisfactory and the user community seems quite friendly and respectful across the platform. I would say it’s worth checking out if you enjoy these types of games. Best regards, and cheers to all. teen patti master login

    Reply
  32. Iwinaviator_vw5

    Technical performance on this iwin aviator platform is realy impressive — loads in under 2 seconds, mobile responsive design is clean and intuitive, zero lag even on 4G connections. UI/UX follows modern web standards with minimal bloat and efficient asset optimization.

    Reply
  34. play plinko

    Personally, I for a while now found how the online plinko market stays booming because people are growing weary of messy features which just slow down this fun. Within recent tests,
    that physical nature behind the plinko app on the handheld device remains truly better rather
    than that computer iteration, primarily as one will directly tap so you drop the sphere.
    Prior to launching any fresh game, ensure
    sure to research play plinko so as to
    find those reliable sites which give a fair plinko gambling game using top multiplier levels.
    My second opinion is how budget management is way times extremely essential
    here as in basic titles because this round stays
    very fast-paced. Does anyone else actually formed a
    consistent method for changing amidst various risk settings, and must
    you believe it is purely down for chance?

    Moreover, which brand will the community reckon shows this most attractive animations at the moment?
    I’ll be happy to hear the comments on the topic!

    Reply
  36. jeu plinko avis

    Pour ma part, j’ai remarqué que le monde du jeu a énormément évolué ces derniers temps, notamment via l’arrivée de nouveautés tels que le plinko.
    Il s’agit d’ un mécanisme super simple lequel procure une solide bouffée d’adrénaline lors de chaque chute du jeton. Selon mes
    tests, le contrôle du capital reste l’élément majeur pour ne pas faire
    faillite trop vite. Il me semble aussi que les démos de plinko
    gratuit constituent un très bon exercice pour tester des stratégies sans pour autant paniquer.
    Toutefois, je me demande souvent quand les codes peuvent être totalement
    transparents pour tout le monde. De votre côté, est-ce que ce genre de jeu va détrôner les jeux
    de table standards ? Je compte sur vous pour connaître votre point de vue à propos
    de ce nouveau style.

    Reply
  37. LudoKing_vy7

    I was skeptical about ludo king gameplay at first because there’s so many copycats out there but I gave it a shot and was genuinely surprised by how polished it feels compared to other versions I’ve tried. The whole experience just feels way more refined than expected honestly.

    Reply
  44. jeux plinko avis

    Honnêtement, il me semble que le paysage des casinos en ligne a énormément
    évolué ces derniers temps. D’après mon expérience,
    pas mal de passionnés cherchent essentiellement de la transparence plutôt
    que les gros bonus. Par exemple, j’ai fréquemment remarqué des avis sur le plinko game avis qui indiquent que
    le gameplay épuré reste un choix gagnant pour se divertir tranquillement.
    À ce propos, si vous souhaitez explorer une plateforme sérieuse, je vous invite à voir
    jeux plinko avis car les graphismes est vraiment propre.
    Néanmoins, je me dis souvent si les RNG sont réellement
    transparents sur tous les sites ou si cela reste une question de pure chance.
    Une autre chose, c’est le rôle des bonus de bienvenue
    qui parfois cachent des exigences assez lourdes. De votre côté,
    préférez-vous viser les retours fréquents réguliers
    ou viser le multiplicateur max en prenant le risque de tout griller en un éclair?
    J’ai hâte de lire vos impressions sur cette question.

    Reply
  46. plinko online game

    I have recently been tracking the way the whole online
    gambling world has transformed quite a bit, particularly concerning these newer
    interactive versions. Truly, finding some reliable plinko game online could
    actually be such the complete mission with all these options nowadays, however this system behind each
    plinko online game honestly assists by showing exactly how a fair result really works during real sessions.
    I realized how a win frequency mostly depends mostly by each peg settings, and this feels
    a point most newbies simply overlook. Tell me, do you feel whether the
    aggressive strategy actually is ever productive long-term, plus do the majority of users mostly like the safer method?

    Furthermore, does anyone else discovered how diverse platforms appear so as to
    have totally varied bounce for the ball? I’d really want and know the views about this.|Why not let us share exactly what you
    actually believe!|I’m looking forward toward a few solid discussion in this thread!|Good luck!|Be lucky!

    Reply
  49. Michaelstaix

    Промокод при регистрации в 1xbet сегодня. Этой акцией может воспользоваться каждый новый игрок при регистрации на сайте букмекера. Это ваше преимущество с начала. В форме регистрации есть необязательное поле для ввода кода. Как получить промокод на 1хбет на дополнительный бонус. В магазине xBonus можно обменять баллы на коды 1xbet kz: для ординаров, экспрессов или лотерей. Найти промокоды 1xbet можно в соцсетях. Представители БК часто публикуют их к праздникам или турнирам. Все актуальные промокоды 2026 доступны в официальном разделе сайта и могут быть обменяны на бонусные баллы.

    Reply
  52. MegaCasino_it6

    I was skeptical about this place at first because you hear so many stories but honestly when i actually started using it the whole process was way smoother than i expected and ive had no issues with withdrawals so far mega typhlosion

    Reply
  62. MegaWin_zt1

    For anyone who needs a platform that doesn’t constantly bombard you with popups and sketchy stuff, MegaWin seems to actually care about user experience. Just do yourself a favor and read the terms before you start, alot of people skip that and then complain about stuff they could’ve understood earlier. game name

    Reply
  64. phuksjKi

    The world of best casino bonuses in Pakistan is rapidly growing in the region right now. You can find a huge variety of slots and table games with high RTP. | New users can often claim generous welcome packages and loyalty rewards. | Professional dealers and high-quality streams create a real atmosphere. ] Don’t miss your chance to win big on a trusted platform.

    Reply
  65. play plinko

    I’ve observed recently the fact that plinko online has definitely become the star of almost all gaming
    circles. It is actually fascinating observing how a game so simple is
    able to generate such excitement just by using each single
    tumble from the top. In my experience, this trick stays choosing a trustworthy plinko online game that
    the math look fair, and I suggest you look at play plinko just to
    get a few good tips ahead of dropping any serious stakes.
    I forever preferred setting that high risk mode because the
    edge slots seem very satisfying if the ball at last strike the target.
    Nevertheless, I’ve still am curious if the number regarding rows actually changes the overall luck, or
    if it’s simply the mental illusion. Which type of settings do everyone normally use during a long run?
    Additionally, do someone here feel the fact that any hour for day
    changes things one bit for the math?

    Reply
  68. phukjpKi

    The world of best Pakistani casino sites offers incredible opportunities for players. You can find a huge variety of slots and table games with high RTP. | Most sites provide mobile-friendly interfaces and 24/7 customer support. | Choosing a licensed operator ensures a fair and transparent gaming process. ] Check the latest rankings and start your journey today!

    Reply
  71. Jax bog

    At the stage of selecting a car I understood that skipping a proper check could lead to serious problems. I began looking for advice and found https://stocar.kh.ua/uk/2025/10/16/yak-pereviriti-avto-pered-pokupkoyu-povnij-chek-list-ta-pokrokova-instruktsiya .
    The article is detailed and explains how to inspect a car before buying. It includes a full checklist that helps you not miss anything.
    After reading, I understood what to check: legal status. This really helps to save money.
    This guide is worth your attention to anyone who wants to feel confident before buying.

    Reply
  74. ValorSlots_ul4

    So my friend told me about this site like two weeks ago and I was like ok whatever but then I actually checked it out and dude the whole thing is pretty smooth, like the navigation is intuitive and I didnt have to dig through a million menus to find what I wanted, anyways im still using it now so ya valor cassino

    Reply
  75. KevinskIse

    Онлайн-торгівля значно спростила процес вибору та придбання різноманітних товарів, включаючи напої для будь-яких подій. Все частіше користувачі шукають інтернет-магазин алкоголю в тетрапаках, щоб знайти вигідні пропозиції без зайвих витрат часу. Це дійсно зручно, коли весь асортимент доступний у кілька кліків.

    Reply
  78. phulaKi

    The current market of best live casinos Pakistan is rapidly growing in the region right now. The security of transactions and fast payouts are guaranteed on these platforms. | Most sites provide mobile-friendly interfaces and 24/7 customer support. | Professional dealers and high-quality streams create a real atmosphere. ] Check the latest rankings and start your journey today!

    Reply
  79. plinko gratuit

    Salut à tous, j’avais envie de soumettre mon point de vue relativement aux changements récentes dans le
    milieu du casino. Sincèrement, j’ai remarqué que les interfaces mettent en avant fréquemment des formats courts assez addictifs, tel que plinko gratuit
    qui paraît se positionner en tant que une référence parmi les
    parieurs. Personnellement, le point capital c’est la
    gestion du budget, car la ferveur tend à rapidement on peut oublier les limites.
    En plus, est-ce que les moteurs de jeu semblent pleinement justes à
    la longue ? Je voudrais lire votre astuce majeure pour ne pas faire banqueroute lors d’une simple partie.
    Et pour vous, quel s’avère être le divertissement qui vous offre le plus
    de fun en 2026 ?

    Reply
  80. phukafKi

    The current market of real money casino Pakistan is rapidly growing in the region right now. You can find a huge variety of slots and table games with high RTP. | Most sites provide mobile-friendly interfaces and 24/7 customer support. | Choosing a licensed operator ensures a fair and transparent gaming process. ] Check the latest rankings and start your journey today!

    Reply
  81. phuktmKi

    Looking at best live casinos Pakistan offers incredible opportunities for players. The security of transactions and fast payouts are guaranteed on these platforms. | Most sites provide mobile-friendly interfaces and 24/7 customer support. | Professional dealers and high-quality streams create a real atmosphere. ] Check the latest rankings and start your journey today!

    Reply
  82. phuknqKi

    The world of best Pakistani casino sites is rapidly growing in the region right now. You can find a huge variety of slots and table games with high RTP. | Most sites provide mobile-friendly interfaces and 24/7 customer support. | Choosing a licensed operator ensures a fair and transparent gaming process. ] Check the latest rankings and start your journey today!

    Reply
  85. phuezKi

    The current market of Pakistani online gambling sites is rapidly growing in the region right now. The security of transactions and fast payouts are guaranteed on these platforms. | Most sites provide mobile-friendly interfaces and 24/7 customer support. | Choosing a licensed operator ensures a fair and transparent gaming process. ] Check the latest rankings and start your journey today!

    Reply
  86. phukeoKi

    Looking at best live casinos Pakistan offers incredible opportunities for players. The security of transactions and fast payouts are guaranteed on these platforms. | New users can often claim generous welcome packages and loyalty rewards. | Professional dealers and high-quality streams create a real atmosphere. ] Check the latest rankings and start your journey today!

    Reply
  87. phukeyKi

    Looking at real money casinos Pakistan offers incredible opportunities for players. You can find a huge variety of slots and table games with high RTP. | New users can often claim generous welcome packages and loyalty rewards. | Choosing a licensed operator ensures a fair and transparent gaming process. ] Check the latest rankings and start your journey today!

    Reply
  90. phukieKi

    The world of real money casinos Pakistan offers incredible opportunities for players. The security of transactions and fast payouts are guaranteed on these platforms. | Most sites provide mobile-friendly interfaces and 24/7 customer support. | Professional dealers and high-quality streams create a real atmosphere. ] Check the latest rankings and start your journey today!

    Reply
  91. MonopolyLive_vh2

    For anyone who needs a heads up, just make sure you understand how monopoli live works before jumping in, took me a minute to figure out the layout but once I got the hang of it things made more sense. Also reccomend reading through their info page so you know what your getting into, saves time and confusion.

    Reply

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top