Skip to main content

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

5
  • I think with Datasets you could use flatMap? Commented Mar 2, 2017 at 14:05
  • 1
    I guess you have to explode your arrays first, then join Commented Mar 2, 2017 at 20:14
  • THe explode functionality is deprecated in Spark 2.1.0 Commented Mar 2, 2017 at 21:10
  • Does DF2 have case class or can you share the creation of `df2'? Commented Mar 3, 2017 at 9:24
  • My mistake the explode functionality is still available in Spark 2.1.0 under functions.explode in the org.apache.spark.sql package Commented Mar 6, 2017 at 14:13