The Wayback Machine - https://web.archive.org/web/20220721133728/https://github.com/topics/databricks
Skip to content
#

databricks

Here are 355 public repositories matching this topic...

SynapseML
brunocous
brunocous commented Sep 2, 2020

I have a simple regression task (using a LightGBMRegressor) where I want to penalize negative predictions more than positive ones. Is there a way to achieve this with the default regression LightGBM objectives (see https://lightgbm.readthedocs.io/en/latest/Parameters.html)? If not, is it somehow possible to define (many example for default LightGBM model) and pass a custom regression objective?

wjones127
wjones127 commented Jul 17, 2022

Description

PyArrow Filesystems don't expect paths to be provided as URIs with a scheme prefix (e.g. s3://). The only reason that DeltaTable.to_pyarrow_dataset() works (if you don't pass a filesystem) is that our DeltaStorageHandler accepts URIs.

We need to change so that:

  1. DeltaStorageHandler rejects scheme prefixes
  2. We always pass file paths to PyArrow functions *withou
bug good first issue binding/python
terraform-provider-databricks
nkvuong
nkvuong commented Jul 20, 2022

Follow-up work for #1460

  • expose name-to-id mapping for warehouses & clusters

Example syntax could be as below

data "databricks_sql_warehouses" "all_shared" {
  warehouse_name_contains = "shared"
}

output "warehouses_mapping" {
  value = data.databricks_sql_warehouses.all_shared.mappings <- this returns { "shared" = "11", "shared_1" = [12, 13]}
}
  • add lookup of cluster
feature good first issue
ApacheSpark

This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.

  • Updated Jun 22, 2022
  • Python

Improve this page

Add a description, image, and links to the databricks topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the databricks topic, visit your repo's landing page and select "manage topics."

Learn more