Disable file creation when Partition does not exist #20133#20151
Conversation
|
@willzgw |
|
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua |
|
Wdyt @codope .. could you review/chime in? Also cc @findepi @brandylove |
|
We also need this merged, since otherwise reading Hudi tables with empty partitions will fail when they are read from a read-only volume. This will then lead to the query being in a stuck state as described in #19506 |
|
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua |
|
Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time. |
|
Since this PR has been open for almost 4 months: What steps are there to do, to get this one merged? We are currently forced to maintain our own fork to apply this fix, but would prefer to go back to directly using the upstream releases. Tagging everyone the github-actions bot tagged 3 weeks ago: @bitsondatadev @colebow @mosabua |
| if (fileIterator.hasNext()) { | ||
| return fileIterator; | ||
| } | ||
| try (OutputStream ignored = metaClient.getFileSystem().newOutputFile(partitionLocation).create()) { |
There was a problem hiding this comment.
I am not sure why this was getting created in the first place. Hudi connector does not support writes. In my opinion, the fix is valid, simply return empty iterator when partition does not exist.
@anusudarsan @homar Do you recall any specific scenario why this was added?
|
@dain would you mind taking a look? |
|
Hi there, I'm desperately waiting for this fix. Is there anything one could do to help getting it merged? |
wendigo
left a comment
wendigo
left a comment
There was a problem hiding this comment.
Can we have a test for that?
|
I don't think a test is required, as this is extra code which shouldn't exist. It seems strange to have a test for "make sure code doesn't randomly create some file". |
10 participants
Description
Additional context and related issues
Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text: