Running Dashboards on Azure Databricks

[achandak33]

Describe the bug
I am trying to run ResponsibleAIDashboard on Azure Databricks Notebook. I have a cluster and I have installed raiwidgets library in the cluster.

When I do this - ResponsibleAIDashboard(rai_insights, cohort_list=cohort_list) , I get the error

ResponsibleAI started at None

I bet it is a IP and port problem because the tutorial - https://github.com/microsoft/responsible-ai-toolbox/blob/main/notebooks/responsibleaidashboard/responsibleaidashboard-census-classification-model-debugging.ipynb- runs seamlessly on my VM or local.

[imatiach-msft]

hi @achandak33 We only support the explanation and fairness dashboards on Databricks (and only the static view for the explanation dashboard). Since Azure Databricks does not allow us to open ports, we can’t actually allow the user to open a new tab and view the all-in-one ResponsibleAI dashboard with dynamic functionality enabled in Azure Databricks.

Please see related closed issue:
#1488

Also, the Error Analysis dashboard has some initial limited support I’ve added a couple months ago to display on Databricks, but only when using a spark model/dataset. I could help you to try to run it.

See related open issue for Error Analysis Dashboard:
#944

Since that issue was opened I have merged this PR to add support for spark models in a static view of the Error Analysis Dashboard:
#1388

Note we don't publicly advertise it on this repository yet as we still need to support the heatmap, add support for categorical columns, fix some minor issues and test the functionality more.