We've moved!
You can find our new documentation site and support forum for posting questions here.

PySpark2 Notebooks in Firecloud to use Hail.

Hello,

I am trying to use Notebooks integrated in the Firecloud. When i see the documentation there was a line that mentioned to use PySpark2 notebooks to use Hail.

Unfortunately when I was tryin to create a notebook there was no PySpark2 Kernel options ( i can see only Python3, Python2 and Hail 0.1) . Any help???

Thanks

Answers

  • AdelaideRAdelaideR Member admin

    @Shiva_gan

    It appears that the Firecloud is not currently supporting Hail 0.2. I have reached out to the development team to see if there is another option in the meantime. Is your data so big that a local install of Hail would not be feasible?

  • AdelaideRAdelaideR Member admin

    @Shiva_gan Here is a workaround:

    When you are creating a Notebook, there is no pyspark option in the Kernel drop down. However if you create that notebook with a Hail o.1 kernel, then open the cluster, in the notebook's Kernel option you can see the Pyspark show up.

Sign In or Register to comment.