Error running Spark on Databricks: constructor public XXX is not whitelisted

I was using Azure Databricks and trying to run some example python code from this page.

But I get this exception:

py4j.security.Py4JSecurityException: Constructor public org.apache.spark.ml.classification.LogisticRegression(java.lang.String) is not whitelisted.

1

Best Answer

Answer recommended by Microsoft AzureCollective

This error shows up with some library methods when using High Concurrency cluster with credential pass through enabled. If that is your scenario a work around that may be an option is to use a different cluster mode.

py4j.security.Py4JSecurityException: ... is not whitelistedThis exception is thrown when you have accessed a method that Azure Databricks has not explicitly marked as safe for Azure Data Lake Storage credential passthrough clusters. In most cases, this means that the method could allow a user on a Azure Data Lake Storage credential passthrough cluster to access another user’s credentials.

Reference: https://learn.microsoft.com/en-us/azure/databricks/data-governance/credential-passthrough/adls-passthrough

Random Posts

How to drop a table if it exists?SQL Sum window functionhow to install geckodriver on a windows systemHow to use Standalone Components, Pipes, or Directives in feature modulesMultiple boolean conditions - operator precedenceWhat is MVC in Ruby on Rails?Multidimensional arrays in Java and C#'abrt-cli status' timed out when login to SSH [closed]What is the differences between Amazon Corretto and Java JDK? Why we use Amazon Corretto for specially developing Enterprise Apps? [closed]Read Numeric Data from a Text File in C++Unable to edit videos recorded in default mode on iPhone 13 mini on iPad 2020 8th generation - iOS v15.5How to fix shadow for menu material-uiTotal memory used by Python process?C: String functions without string.h libraryBlank option in filter. But where is it coming from?CRT file does not contain a privatekey [closed]Using str_sub from stringr to replace characters in all elements of a vectorHow to install Python packages for SpyderHow to install or upgrade in ansible all packages in a yum repositoryHow to turn off bash's "readline arguments" for character repetition?